linux-kernel - Re: [kernel.org users] XZ Migration discussion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 13 Feb 2010 11:30:07 -0800
From:	Randy Dunlap <rdunlap@...otime.net>
To:	Geert Uytterhoeven <geert@...ux-m68k.org>
CC:	Jean Delvare <khali@...ux-fr.org>, Pavel Machek <pavel@....cz>,
	"J.H." <warthog9@...nel.org>,
	"FTPAdmin Kernel.org" <ftpadmin@...nel.org>, users@...nel.org,
	lasse.collin@...aani.org,
	linux-kernel <linux-kernel@...r.kernel.org>, mirrors@...nel.org
Subject: Re: [kernel.org users] XZ Migration discussion

On 02/13/10 10:49, Geert Uytterhoeven wrote:
> On Sat, Feb 13, 2010 at 18:10, Jean Delvare <khali@...ux-fr.org> wrote:
>> On Thu, 11 Feb 2010 21:51:29 +0100, Pavel Machek wrote:
>>> I believe this is cleanest. gzip has performance advantages, (...)
>>
>> I have been investigating this issue and would like to share my
>> findings as an additional data point for this discussion.
>>
>> For my testing, I have been using the slowest machine I still have
>> available here: a Pentium 166 MMX, with 64 MB of memory and a slow hard
>> disk drive. I've been writing down the duration of each task it took to
>> boot kernel 2.6.27.45 on this machine. I did this for both .gz and .bz2
>> formats.
>>
>> Raw results are as follow (format=min:s):
>>
>> downloading linux-2.6.27.tar.bz2          5:01
>> downloading patch-2.6.27.45.bz2           0:02
>> unpacking linux-2.6.27.tar.bz2            7:28
>> applying patch-2.6.27.45.bz2              1:21
>> ----------------------------------------------
>> total for bz2                            13:52
>>
>> downloading linux-2.6.27.tar.gz           6:23
>> downloading patch-2.6.27.45.gz            0:02
>> unpacking linux-2.6.27.tar.gz             3:20
>> applying patch-2.6.27.45.gz               1:10
>> ----------------------------------------------
>> total for gz                             10:55
>>
>> So the gz option is unsurprisingly faster, setting up the source tree
>> takes almost 3 minutes less (-21%).
>>
>> Then the (common) build and installation times:
>>
>> building                                117:26
>> installing modules                        0:12
>> ----------------------------------------------
>> total                                   117:38
>>
>> This is a customized kernel, as small as I could do, with almost no
>> features and the minimal set of drivers. As you can see, the build time
>> is one order of magnitude greater than the tree setup time. Comparing
>> the total times from download to install between bz2 and gz:
>>
>> bz2: 13:52 + 117:38 = 131:30
>> gz:  10:55 + 117:38 = 128:33
>>
>> Compared to bz2, gz saves... 2% on the overall time. As a conclusion, I
>> think we can plain discard the argument "I need .gz because my machine
>> is slow" from now on. It simply doesn't hold.
> 
> 166 MHz and 64 MB of RAM is still an order of magnitude more than some other
> machines Linux is capable of running on.
> 
> Of course we no longer build kernels on those machines natively, we
> cross-compile
> on much faster machines (and use git to fetch the kernel sources, FWIW ;-).
> 
> BTW, who still downloads full kernel tarballs? From my experience, that's mostly
> distro people who have scripts to build kernels from tarballs + a
> bunch of patches?

All of my daily build & boot testing downloads tarballs.
They use full linux-x.y.z for "releases" (like 2.6.32)
and they use patch-x.y.z.gz/bz2 for intermediate versions.

I don't mind updating the scripts to use .xz tarballs, but I would
really prefer to stick with tarballs instead of git trees.

> I guess actual developers use git nowadays, even if they're stuck on
> an old 2.6.x kernel?
> Backporting critical fixes is sooo much easier using git cherry-pick...
> 
> Do we have statistics about tarball downloads vs. git clone / git pull?

That would be good to see.

-- 
~Randy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/