linux-kernel - Re: Kernel compression benchmarks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAODFU0ogKh6iz_F9fOYwqChfWZ+VJXoBebzzMHiOtdTK_jXqqg@mail.gmail.com>
Date:   Sun, 26 Jul 2020 18:43:34 +0200
From:   Jan Ziak <0xe2.0x9a.0x9b@...il.com>
To:     alex_y_xu@...oo.ca
Cc:     Kernel-team@...com, clm@...com, gregkh@...uxfoundation.org,
        keescook@...omium.org, kilobyte@...band.pl,
        linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        mingo@...nel.org, nickrterrell@...il.com, nolange79@...il.com,
        oss@...at.biz, patrick@...cx.xyz, patrickw3@...com, rmikey@...com,
        sedat.dilek@...il.com, terrelln@...com, x86@...nel.org
Subject: Re: Kernel compression benchmarks

Hello

I looked at the SVG graphs and it appears that the formula used wasn't
T_load+T_decompress, but was just T_decompress.

Without considering the time it takes to load the compressed data from
a storage device, the SVG graphs are only half-done and might be
deceiving.

There are 3 kinds of typical device speeds nowadays, the sequential
read speed of a large non-fragmented compressed file is one of the
following:

100 MB/s: rotational disks and USB flash drives
500 MB/s: SATA SSD
2 GB/s: NVMe SSD

The read speeds of USB flash devices vary a lot, but in case of recent
high-speed USB flash drives it falls into the 100 MB/s category of
rotational disks. Taking USB flash read speed into consideration is
important for deciding which compression to use when creating the ISO
image of a Linux distribution.

In summary: Instead of the 1 kernel-decomp.svg file, there should be 3
kernel-read-decomp.svg files. Similarly in the case of the
initramfs-decomp.svg file.

As a rule of thumb, if the kernel and initramfs are stored on a NVMe
SSD then simply select the fastest decompressor without considering
the compression ratio - or avoid using any compression at all in which
case T_decompress will be zero.

The formula T_load+T_decompress assumes that loading and decompression
aren't executing in parallel. If they are, the formula should be
max(T_load, T_decompress).

Sincerely
Jan