linux-kernel - Re: [kernel.org users] XZ Migration discussion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100216140004.GB4821@elf.ucw.cz>
Date:	Tue, 16 Feb 2010 15:00:04 +0100
From:	Pavel Machek <pavel@....cz>
To:	Jean Delvare <khali@...ux-fr.org>
Cc:	mirrors@...nel.org, lasse.collin@...aani.org,
	linux-kernel <linux-kernel@...r.kernel.org>, users@...nel.org,
	"FTPAdmin Kernel.org" <ftpadmin@...nel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Willy Tarreau <w@....eu>
Subject: Re: [kernel.org users] XZ Migration discussion


> > > I have an old, slow machine here which I am going to use to perform
> > > some real world testing, and I'll post the results when I'm done. But I
> > > suspect that building a kernel on this machine, even a small one with
> > > just the drivers it needs, will take much longer than unpacking the
> > > sources. So anyone worrying about performance would rather rely on
> > > cross-compilation, and in turn can afford whatever decompression tool
> > > is needed.
> > 
> > On zaurus, kernel compilation takes 4 hours. (I.e. "one night"). So
> > that one is ... well ... done overnight.
> 
> Out of curiosity, if it takes that long, why don't you use a
> cross-compiler?

Because I hack it on the go. First build is long and ugly, but
subsequent builds only take 5 minutes or so, so development is
possible.

(If I crosscompiled it, I'd have to crosscompile even the subsequent
builds, which is impossible -- no powerful machine nearby).

> > Untar is something I normally wait for, since you need to run
> > (interactive) oldconfig after that.
> 
> You'll have to wait, no matter what compression format you use (and
> even if you don't compress the tarball). Judging by the duration of the
> build on your machine, I'd estimate the decompression time to 7 minutes
> for gz vs. 15 minutes for bz2 maybe? I doubt you sit in front of the
> machine for 7 minutes waiting for tar.gz to decompress, right? So I
> fail to see what difference it makes. You'll just do something else for
> 15 minutes instead of doing something else for 7 minutes.

Actually I do sit in front of the machine, reading mails while it
decompresses.

[I'll get some numbers.]

sh-3.2$ time bzip2 -d < ~/.ketchup/l^Ginux-2.6.31.tar.bz2 > delme.tar
485.73user 137.35system 683.32 (11m23.320s) elapsed 91.18%CPU
sh-3.2$ df -h^H^H^H^H^Htime cat delme.tar > /usr/src/delme.tar
0.57user 109.03system 381.13 (6m21.133s) elapsed 28.75%CPU
sh-3.2$ time zca^G^Gt delme.tar > /u ^H^H ^H^H ^H^H ^H^Hdelme.tar.gz
^H^H ^H^H ^H^H
+^H^H ^H^H ^H^H ^H^H ^H^H ^H^H ^H^H ^H^H ^H^H ^H^H.gz > delme.tar
43.97user 106.22system 223.26 (3m43.261s) elapsed 67.27%CPU

So... gzip is actually _faster_ than uncompressed data, while bzip is
twice slower. Don't know about xzip.

Anyway, gzip just makes sense. It is both smaller and faster than
alternatives, and nearly as portable.

> Anyway, as I have been saying several times already, nothing prevents
> you from repacking tarballs to gz before uploading it to your slow
> system if such is your desire. I can understand the portability
> argument, but the decompression time, no way.

Ok, so lets go by the portability argument.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/