linux-kernel - Re: About support XZ-compressed kernel on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160215222622.7e159519@tukaani.org>
Date:	Mon, 15 Feb 2016 22:26:22 +0200
From:	Lasse Collin <lasse.collin@...aani.org>
To:	Baoquan He <bhe@...hat.com>
Cc:	linux-kernel@...r.kernel.org, hpa@...or.com, alain@...ff.lu,
	phillip@...gher.demon.co.uk, akpm@...ux-foundation.org,
	keescook@...omium.org, bp@...en8.de, vgoyal@...hat.com
Subject: Re: About support XZ-compressed kernel on x86

On 2016-02-14 Baoquan He wrote:
> On 02/13/16 at 08:57pm, Lasse Collin wrote:
> > The long comment in arch/x86/boot/compressed/misc.c explains the
> > need for the offset for gzip/Deflate. A similar comment in
> > lib/decompress_unxz.c explains it for XZ/LZMA2.  
> 
> Thank you so much, Lasse. You clearly pointed out my confusion.
> Yeah, I didn't understand it well. Your description for xz in
> lib/decompress_unxz.c is very helpful. The 64K is the maximum payload
> in one chunk. Adding this 64K is to avoid the worst case that very
> small payload can reprsent a 64K uncompressed output data. With my
> understanding it could be  a chunk which contains complete duplicate
> content. like all "0" or other stuff?

Yes, like all zeros. I wrote another explanation just in case it helps:

In-place decompression puts the compressed data at the end of the
buffer and decompresses it to the beginning of the buffer:

    F = free memory
    K = uncompressed kernel
    C = compressed input data

    Start:         FFFFFFFFFFFFFFCCCCCCCC
    Decompressing: KKKKKKKKKKFFFFFFFFCCCC
    Finished:      KKKKKKKKKKKKKKKKKKKKFF

The free memory (FF) at the end is the safety margin.

In the worst case the beginning of the uncompressed data compresses to
almost nothing (like all zeros do), and the end of the data is
incompressible. In the beginning the write position of the decompressor
advances quickly while the read position moves very little, and thus
the write position quickly approaches the read position:

    Start:         FFFFFFFFFFFFFFCCCCCCCC
    Decompressing: KKKKKKKKKKKKKKFCCCCCCC
    Finished:      KKKKKKKKKKKKKKKKKKKKFF

The safety margin ensures that the write position can never overtake
the read position.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode