linux-kernel - Re: About support XZ-compressed kernel on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 16 Feb 2016 21:20:20 +0800
From:	Baoquan He <bhe@...hat.com>
To:	Lasse Collin <lasse.collin@...aani.org>
Cc:	linux-kernel@...r.kernel.org, hpa@...or.com, alain@...ff.lu,
	phillip@...gher.demon.co.uk, akpm@...ux-foundation.org,
	keescook@...omium.org, bp@...en8.de, vgoyal@...hat.com
Subject: Re: About support XZ-compressed kernel on x86

On 02/15/16 at 10:26pm, Lasse Collin wrote:
> On 2016-02-14 Baoquan He wrote:
> > On 02/13/16 at 08:57pm, Lasse Collin wrote:
> > > The long comment in arch/x86/boot/compressed/misc.c explains the
> > > need for the offset for gzip/Deflate. A similar comment in
> > > lib/decompress_unxz.c explains it for XZ/LZMA2.  
> > 
> > Thank you so much, Lasse. You clearly pointed out my confusion.
> > Yeah, I didn't understand it well. Your description for xz in
> > lib/decompress_unxz.c is very helpful. The 64K is the maximum payload
> > in one chunk. Adding this 64K is to avoid the worst case that very
> > small payload can reprsent a 64K uncompressed output data. With my
> > understanding it could be  a chunk which contains complete duplicate
> > content. like all "0" or other stuff?
> 
> Yes, like all zeros. I wrote another explanation just in case it helps:

Yes, this is great and very helpful for people who want to understand
this details. I want to make some change to improve the readability of
the description in boot/compressed/misc.c, do you mind if I put these
there? Or you can post a patch to adjust it.

Thanks
Baoquan

> 
> In-place decompression puts the compressed data at the end of the
> buffer and decompresses it to the beginning of the buffer:
> 
>     F = free memory
>     K = uncompressed kernel
>     C = compressed input data
> 
>     Start:         FFFFFFFFFFFFFFCCCCCCCC
>     Decompressing: KKKKKKKKKKFFFFFFFFCCCC
>     Finished:      KKKKKKKKKKKKKKKKKKKKFF
> 
> The free memory (FF) at the end is the safety margin.
> 
> In the worst case the beginning of the uncompressed data compresses to
> almost nothing (like all zeros do), and the end of the data is
> incompressible. In the beginning the write position of the decompressor
> advances quickly while the read position moves very little, and thus
> the write position quickly approaches the read position:
> 
>     Start:         FFFFFFFFFFFFFFCCCCCCCC
>     Decompressing: KKKKKKKKKKKKKKFCCCCCCC
>     Finished:      KKKKKKKKKKKKKKKKKKKKFF
> 
> The safety margin ensures that the write position can never overtake
> the read position.
> 
> -- 
> Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode