linux-kernel - Re: tip: bzip2/lzma now in tip:x86/setup-lzma

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <499BBD62.5090506@knaff.lu>
Date:	Wed, 18 Feb 2009 08:48:50 +0100
From:	Alain Knaff <alain@...ff.lu>
To:	"H. Peter Anvin" <hpa@...or.com>
CC:	Ingo Molnar <mingo@...e.hu>, Jan Engelhardt <jengelh@...ozas.de>,
	the arch/x86 maintainers <x86@...nel.org>,
	linux-kernel@...r.kernel.org
Subject: Re: tip: bzip2/lzma now in tip:x86/setup-lzma

H. Peter Anvin wrote:
> Ingo Molnar wrote:
>> still fails:
>>
>> [    3.196037] initcall inet_init+0x0/0x1e9 returned 0 after 101777 usecs           
>> [    3.202886] calling  af_unix_init+0x0/0x55 @ 1                                   
>> [    3.207461] NET: Registered protocol family 1                                    
>> [    3.211950] initcall af_unix_init+0x0/0x55 returned 0 after 4394 usecs           
>> [    3.218624] calling  populate_rootfs+0x0/0xd2 @ 1                                
>> [    3.223460] Kernel panic - not syncing: compression method gzip not configured   
>>
>> config attached.
>>
> 
> Okay, I think I've tracked down the problem.
> 
> The issue is that usr/Makefile doesn't correctly filter compression 
> methods which we don't support.  This particular case will happen when 
> you're building with Ingo's attached configuration *and no bzip2 tool 
> installed on the system*.

Weird. When I tried to reproduce this, I got the following error
instead, already during compilation:

  GEN     usr/initramfs_data.cpio.bz2
/home/alain/Projects/Udpcast/Kernels/linux.trees.git/scripts/gen_initramfs_list.sh:
line 295: bzip2: command not found
make[2]: *** [usr/initramfs_data.cpio.bz2] Error 127
make[1]: *** [usr] Error 2
make: *** [sub-make] Error 2

hmmm, but on a second glance, there seems to be indeed an issue. When
compiling it a _second time_, compilation will "succeed" but produce a
bad kernel because the first attempt left an empty
usr/initramfs_data.cpio.bz2 .

I'll look into it this evening, probably a case of "bad" failure
recovery in the Makefile.

> Basically, the algorithm used in usr/Makefile is simply wrong.

Wow, those are mightily strong words...

> The claim is:
> 
> +# Find out "preferred" ramdisk compressor. Order of preference is
> +#  1. bzip2 efficient, and likely to be present
> +#  2. gzip former default
> +#  3. lzma
> +#  4. none
> +
> +# None of the above
> +suffix_y                   =
> +
> +# Lzma, but no gzip nor bzip2
> +suffix_$(CONFIG_RD_LZMA)   = .lzma
> +
> +# Gzip, but no bzip2
> +suffix_$(CONFIG_RD_GZIP)   = .gz
> +
> +# Bzip2
> +suffix_$(CONFIG_RD_BZIP2)  = .bz2
> 
> This doesn't end up make any sense.

Huh? Can't we just stay polite here...

>  The sanest thing to do is to 
> compress with all the compressors that are configured into the kernel, 
> and then use the smallest image of the ones that can be produced, 
> *including the uncompressed image*.

... so, if I understood correctly, that would mean compressing the
kernel 4 times, and keeping the smallest?
Given the relatively low speed of compression, I feel that such a
feature would really annoy people who compile their kernel often (such
as kernel or module developers). Indeed, when only changing a single
file, kernel compression is the most time-consuming part of recompilation.

>  Including the uncompressed image 
> will guarantee that we always have a working option.

Indeed, Ingo's example had the following config:
# CONFIG_RD_GZIP is not set
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y

So, in his case an uncompressed builtin ramdisk would indeed: be the
only solution because
 1. gzip is not configured
 2. bzip2 is not present on build system as per hypothesis
 3. lzma being more "exotic" than bzip2 would probably be absent too.

> 	-hpa

Maybe another solution would be to make the choice of builtin ramdisk
compression user-selectable, and default to no compression at all.

Indeed, in the default case, the builtin ramdisk is so small (950 bytes
uncompressed), that it probably wouldn't really matter anyways.

The only case where it matters is for developers of embedded systems who
want to replace the builtin ramdisk with a fully populated one, because
their boot loader does not support loading a "normal" initrd.

These people are (hopefully) knowledgeable enough to pick an appropriate
compressor (but there's still the issue of notifying them about the
change, obviously).

Btw, what *is* the standard work flow of supplying your own built-in
initramfs? Do such developers usually supply a directory tree, or do
they already cpio it before supplying it to the kernel? Or do they even
compress it themselves?

Regards,

Alain
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/