linux-kernel - Re: request baset device mapper in Linux

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Thu, 06 Oct 2011 14:11:39 +0900
From:	"Jun'ichi Nomura" <j-nomura@...jp.nec.com>
To:	Lukas Hejtmanek <xhejtman@....muni.cz>
CC:	Mike Snitzer <snitzer@...hat.com>,
	Kiyoshi Ueda <k-ueda@...jp.nec.com>, agk@...hat.com,
	linux-kernel@...r.kernel.org
Subject: Re: request baset device mapper in Linux

Hi Lukas,

On 10/05/11 19:35, Lukas Hejtmanek wrote:
> On Wed, Oct 05, 2011 at 05:13:36PM +0900, Jun'ichi Nomura wrote:
>>> yes, 3GB/s and only kwapd0 and kswapd1 is running, no kworker or ksoftirqd..
>>
>> Hmm.. did you find any difference in your profile this time?
> 
> not sure what do you mean.

With SLES 2.6.32.36-0.5-default kernel, you found ksoftirqd
spent most of the time in __blk_recalc_rq_segments, using
sysprof/oprofile. That's why my patch was effective for it.

My question is whether you see such difference between
no-multipath and multipath, in profile data of 3.0.3 (without my patch).

>> I'm trying to reproduce it myself but no success so far
>> (perhaps disks are not fast enough to saturate CPU on my test machine).
> 
> hmm, I have 80 SAS 2.0 disks and two E5640 @ 2.67GHz cpus. 
>  
>> As ksoftirqd in top implies your CPU4 gets too much I/O completions,
>> 'rq_affnity = 2' for both dm and SCSI devices might be a solution.
>> It'll distribute block completion softirqs to submitters and possibly
>> reduce the loads of the 1st CPU in the socket.
>> (See the commit below. It's a new feature of 3.1. Not available in 3.0...)
> 
> So what next? Should I try 3.1 kernel with this patch applied?

Please try 3.1 (without my patch) + 'rq_affinity = 2',
on both multipath and no-multipath.

If you still see performance difference and ksoftirqd
spends most of the time in __blk_recalc_rq_segments,
try 3.1 + my patch + 'rq_affinity = 2'.

Thanks,
-- 
Jun'ichi Nomura, NEC Corporation
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/