linux-kernel - Re: About support XZ-compressed kernel on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160214133138.GA2552@x1.redhat.com>
Date:	Sun, 14 Feb 2016 21:31:38 +0800
From:	Baoquan He <bhe@...hat.com>
To:	Lasse Collin <lasse.collin@...aani.org>
Cc:	linux-kernel@...r.kernel.org, hpa@...or.com, alain@...ff.lu,
	phillip@...gher.demon.co.uk, akpm@...ux-foundation.org,
	keescook@...omium.org, bp@...en8.de, vgoyal@...hat.com
Subject: Re: About support XZ-compressed kernel on x86

On 02/13/16 at 08:57pm, Lasse Collin wrote:
> On 2016-02-12 Baoquan He wrote:
> > Now I have a question about the commit from you:
> > 
> > commit 303148045aac34b70db722a54e5ad94a3a6625c6
> > Author: Lasse Collin <lasse.collin@...aani.org>
> > Date:   Wed Jan 12 17:01:24 2011 -0800
> > 
> >     x86: support XZ-compressed kernel
> > 
> > 
> > In this commit for adding support of XZ-compressed kernel on x86, you
> > add extra 32K to the extract_offset. In commit log you said this is
> > because "The XZ decompressor needs around 30 KiB of heap, so the heap
> > size is increased to 32 KiB on both x86-32 and x86-64." With my
> > understanding decompression is done in decompression stage and it uses
> > boot_heap in arch/x86/boot/compressed/head_64.S, and boot_heap is
> > assigned to free_mem_ptr which is used for decompression heap malloc.
> > During this decompressio stage it's still in copied ZO space, why did
> > you add extra 32K space to extract_offset?  If you want to increase
> > the decompression heap space shouldn't you decrease the
> > extract_offset? Do I misunderstand anything or miss things?
> 
> The reason to increase the heap size in arch/x86/include/asm/boot.h is
> unrelated to the reason why the offset was changed in
> arch/x86/boot/compressed/mkpiggy.c.
> 
> The long comment in arch/x86/boot/compressed/misc.c explains the need
> for the offset for gzip/Deflate. A similar comment in
> lib/decompress_unxz.c explains it for XZ/LZMA2.

Thank you so much, Lasse. You clearly pointed out my confusion.
Yeah, I didn't understand it well. Your description for xz in
lib/decompress_unxz.c is very helpful. The 64K is the maximum payload in
one chunk. Adding this 64K is to avoid the worst case that very small
payload can reprsent a 64K uncompressed output data. With my
understanding it could be  a chunk which contains complete duplicate
content. like all "0" or other stuff?

Thanks
Baoquan

> 
> Smaller safety-margins can work in practice since the calculated
> margins are for the worst case. I'm not even sure if such calculations
> have been done for the other decompressors in Linux.