linux-kernel - RE: scheduling oddity on 2.6.20.3 stock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Pine.LNX.4.64.0705161247490.1280@asgard.lang.hm>
Date:	Wed, 16 May 2007 12:49:42 -0700 (PDT)
From:	david@...g.hm
To:	David Schwartz <davids@...master.com>
cc:	linux-kernel@...r.kernel.org
Subject: RE: scheduling oddity on 2.6.20.3 stock

On Thu, 3 May 2007, David Schwartz wrote:

>> I needed to recompress some files from .bz2 to .gz so I setup a script to
>> do
>>
>> bunzip2 -c $file.bz2 |gzip -9 >$file.gz
>>
>> I expected that the two CPU heavy processes would end up on different
>> cpu's and spend a little time shuffling data between the two cpu's on a
>> system (dual core opteron)
>>
>> however, instead what I find is that each process is getting 50% of one
>> cpu while the other cpu is 97% idle.
>
> That would only be possible if the compression/decompression block size is
> small compared to the maximum pipe buffer size. I suspect the reverse is the
> case.

I'm still running into this problem in various forms

is there an easy way to change the maximum pipe buffer size? (including a 
simple change to the kernel source, I do compile my own kernels)

> It would be interesting to write an intermediate process that basically
> enlarged the pipe buffers and see if that changed anything. Basically, the
> intermediate process would allocate a large buffer (16MB or so) and fill it
> from 'bunzip2' while draining it to 'gzip' in a non-blocking way (unless the
> buffer was full/empty, of course).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/