linux-kernel - Re: [PATCH] init: bzip2 or lzma -compressed kernels and initrds

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <200809142037.48204.rob@landley.net>
Date:	Sun, 14 Sep 2008 20:37:47 -0500
From:	Rob Landley <rob@...dley.net>
To:	Willy Tarreau <w@....eu>
Cc:	Alain Knaff <alain@...ff.lu>, torvalds@...ux-foundation.org,
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH] init: bzip2 or lzma -compressed kernels and initrds

On Sunday 07 September 2008 00:48:31 Willy Tarreau wrote:
> Hi Alain,
> > +config KERNEL_LZMA
> > +       bool "LZMA"
> > +       help
> > +         The most recent compression algorithm.
> > +	 Its ratio is best, decompression speed is between the other
> > +	 2. Compression is slowest.
> > +	 The kernel size is about 33 per cent smaller with lzma,
> > +	 in comparison to gzip.
>
> isn't memory usage in the same range as bzip2 ?

Last I checked it was more.  (I very vaguely recall somebody saying 16 megs 
working space back when this was first submitted to busybox, but that was a 
few years ago...)

A quick Google found a page that benchmarks them.  Apparently it depends 
heavily on which compression option you use:

http://tukaani.org/lzma/benchmarks

Something compressed with lzma -6 takes 5 megabytes to decompress, -7 takes 9 
megs, -8 takes 17 megs, and -9 takes 33.  (Plus your source and target 
buffers for in-memory compression.)

So decompressing anything more than an "allnoconfig" kernel with lzma -6 
probably wouldn't quite fit in 8 megs (800k source data, 2.5 megs destination 
data, 5 megs working space... boing.).  You'd need to bump up to 12 or so.  
And compressed with -7 or -8 you're talking 32 megs or more.

> > +# Bzip2
> > +#
> > -------------------------------------------------------------------------
> >-- +
> > +# Bzip2 does not include size in file... so we have to fake that
> > +size_append=perl -e 'print(pack("i",(stat($$ARGV[0]))[7]));'
> > +
> > +quiet_cmd_bzip2 = BZIP2    $@
> > +cmd_bzip2 = (bzip2 -9 < $< ; $(size_append) $<) > $@
>
> It would be the only command in the build process that requires perl. While
> not a disaster, it's a bit annoying to introduce a new build dependency.

While I despise requiring perl to build the kernel, I'd like to point out that 
H. Peter Anvin introduced Perl as a build requirement for 2.6.25:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=bdc807871d58285737d50dc6163d0feb72cb0dc2

Before that perl was only required for optional things like bloat-o-meter, not 
to actually _build_ the kernel.

My argument about it with him and Sam Ravnborg back in February was at:
http://lkml.org/lkml/2008/2/15/541

And their position was "perl is inevitable, go ahead and add python too if you 
like".  Apparently your build environment can have an infinite number of 
requirements, including things like perl where the implementation is the 
spec.  (There is no perl spec, there's only a perl implementation.  Python at 
least has a java implementation...)

My updated patch to remove the dependency from 2.6.26 was here:
http://www.mail-archive.com/linux-embedded%40vger.kernel.org/msg00748.html

Rob
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/