linux-kernel - RE: scheduling oddity on 2.6.20.3 stock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <MDEHLPKNGKAHNMBLJOLKMEIIDNAC.davids@webmaster.com>
Date:	Wed, 16 May 2007 13:59:43 -0700
From:	"David Schwartz" <davids@...master.com>
To:	"David Schwartz" <davids@...master.com>, <david@...g.hm>
Cc:	<linux-kernel@...r.kernel.org>
Subject: RE: scheduling oddity on 2.6.20.3 stock

> > >> bunzip2 -c $file.bz2 |gzip -9 >$file.gz

So here are some actual results from a dual P3-1Ghz machine (2.6.21.1,
CFSv9). First lets time each operation individually:

$ time bunzip2 -k linux-2.6.21.tar.bz2

real    1m5.626s
user    1m2.240s
sys     0m3.144s

$ time gzip -9 linux-2.6.21.tar

real    1m17.652s
user    1m15.609s
sys     0m1.912s

The compress was the most complex (no surprise there) but they are close
enough that efficient overlap will definitely affect the total wall time. If
we can both decompress and compress in 1:17, we are optimal. First, let's
try the normal way:

$ time (bunzip2 -c linux-2.6.21.tar.bz2 | gzip -9 > test1)

real    1m45.051s
user    2m16.945s
sys     0m2.752s

1:45, or 1/3 over optimal. Now, with a 32MB non-blocking cache between the
two processes ('accel' creates a 32MB cache and uses 'select' to fill from
stdin and empty to stdout without blocking either direction):

$ time (bunzip2 -c linux-2.6.21.tar.bz2 | ./accel | gzip -9 > test2)

real    1m18.361s
user    2m19.589s
sys     0m6.356s

Within testing accuracy of optimal.

So it's not the scheduler. It's the fact that bunzip2/gzip have inadequate
input/output buffering. I don't think it's unreasonable to consider this a
defect in those programs.

DS

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/