linux-kernel - RE: scheduling oddity on 2.6.20.3 stock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <MDEHLPKNGKAHNMBLJOLKCEIIDNAC.davids@webmaster.com>
Date:	Wed, 16 May 2007 13:06:28 -0700
From:	"David Schwartz" <davids@...master.com>
To:	<david@...g.hm>
Cc:	<linux-kernel@...r.kernel.org>
Subject: RE: scheduling oddity on 2.6.20.3 stock


> On Thu, 3 May 2007, David Schwartz wrote:
>
> >> I needed to recompress some files from .bz2 to .gz so I setup
> a script to
> >> do
> >>
> >> bunzip2 -c $file.bz2 |gzip -9 >$file.gz
> >>
> >> I expected that the two CPU heavy processes would end up on different
> >> cpu's and spend a little time shuffling data between the two cpu's on a
> >> system (dual core opteron)
> >>
> >> however, instead what I find is that each process is getting 50% of one
> >> cpu while the other cpu is 97% idle.
> >
> > That would only be possible if the compression/decompression
> block size is
> > small compared to the maximum pipe buffer size. I suspect the
> reverse is the
> > case.
>
> I'm still running into this problem in various forms
>
> is there an easy way to change the maximum pipe buffer size? (including a
> simple change to the kernel source, I do compile my own kernels)

No. Changing the size will not do what you want it to do since that only
tells the kernel what the size is, it does not determine what it is.

> > It would be interesting to write an intermediate process that basically
> > enlarged the pipe buffers and see if that changed anything.
> > Basically, the
> > intermediate process would allocate a large buffer (16MB or so)
> > and fill it
> > from 'bunzip2' while draining it to 'gzip' in a non-blocking
> > way (unless the
> > buffer was full/empty, of course).

It is not particularly hard to write such a process. I have a proxy that I
can easily tweak to do this. I'm going to give it a shot and see if it
helps.

DS


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/