linux-kernel - Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180817175746.d2brxipp7xa4y2uw@angband.pl>
Date:   Fri, 17 Aug 2018 19:57:46 +0200
From:   Adam Borowski <kilobyte@...band.pl>
To:     Andi Kleen <ak@...ux.intel.com>
Cc:     René Rebe <rene@...ctcode.com>,
        Nick Terrell <terrelln@...com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>,
        "H. Peter Anvin" <hpa@...or.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "x86@...nel.org" <x86@...nel.org>,
        Kernel Team <Kernel-team@...com>, Chris Mason <clm@...com>,
        Julian Andres Klode <julian.klode@...onical.com>
Subject: Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

On Fri, Aug 17, 2018 at 09:54:03AM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 06:15:45PM +0200, René Rebe wrote:
> > Hey,
> > 
> > is there any mainline future for this zstd support?
> > Currently my most favourite compressor for this, and for what it’s worth
> > zstd/initrd now even tested on #t2sde / hp/pa-risc
> 
> Can you remove some other compressor for the kernel image if you merge this?

Funny you say this.  I want to post this after the merge window is over:
https://github.com/kilobyte/linux/commits/nobz2-next-20180731

Sorry for a tree atop next, but there are conflicts that needed to be
adressed.  The patchset is also undertested (works for me, but not even
compile-tested on archs I don't have a toolchain for at hand).

> The "favourite compressor" seems to roughly change every year, so if
> we keep adding new ones things will get more and more convoluted.

The above patchset drops just bzip2.  It is the only one that's strictly
beaten in every way (ratio, time, memory usage), there are also no other
uses of bzip2 anywhere in the kernel so we'd get to drop its code
completely: 900 lines of Linus' happiness.

Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
but those are used elsewhere thus there's hardly any gain.  If you want them
gone, please say so -- I'll include their droppage.

> Surely zstd is universally better than some existing compressor
> the kernel already uses for itself for a particular sweet spot?
> If it isn't it's likely not needed.

zstd is better in the balanced niche: unless you want extreme speed (lz4) or
best compression (xz), zstd handily beats algorithms in the middle.

Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄⠀⠀⠀⠀ • use glitches to walk on water [Mt14:25-26]