linux-kernel - Re: [PATCH v2 0/4] ipc: reduce ipc lock contention

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <51364AB9.80206@hp.com>
Date:	Tue, 05 Mar 2013 14:42:49 -0500
From:	Waiman Long <Waiman.Long@...com>
To:	Rik van Riel <riel@...hat.com>
CC:	Davidlohr Bueso <davidlohr.bueso@...com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Emmanuel Benisty <benisty.e@...il.com>,
	"Vinod, Chegu" <chegu_vinod@...com>,
	"Low, Jason" <jason.low2@...com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	"H. Peter Anvin" <hpa@...or.com>,
	Andrew Morton <akpm@...ux-foundation.org>, aquini@...hat.com,
	Michel Lespinasse <walken@...gle.com>,
	Ingo Molnar <mingo@...nel.org>,
	Larry Woodman <lwoodman@...hat.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Steven Rostedt <rostedt@...dmis.org>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH v2 0/4] ipc: reduce ipc lock contention

On 03/05/2013 12:10 PM, Rik van Riel wrote:
> On 03/05/2013 04:35 AM, Davidlohr Bueso wrote:
>
>> 2) While on an Oracle swingbench DSS (data mining) workload the
>> improvements are not as exciting as with Rik's benchmark, we can see
>> some positive numbers. For an 8 socket machine the following are the
>> percentages of %sys time incurred in the ipc lock:
>>
>> Baseline (3.9-rc1):
>> 100 swingbench users: 8,74%
>> 400 swingbench users: 21,86%
>> 800 swingbench users: 84,35%
>>
>> With this patchset:
>> 100 swingbench users: 8,11%
>> 400 swingbench users: 19,93%
>> 800 swingbench users: 77,69%
>
> Does the swingbench DSS workload use multiple semaphores, or
> just one?
>
> Your patches look like a great start to make the semaphores
> more scalable. If the swingbench DSS workload uses multiple
> semaphores, I have ideas for follow-up patches to make things
> scale better.
>
> What does ipcs output look like while running swingbench DSS?
>

For Oracle, the semaphores are set up when the instance is started 
irrespective of the workload. For a 8-socket 80 cores test system, the 
output of ipcs look like:

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x00000000 0          root       600        1
0x00000000 65537      root       600        1
0xcd9652f0 4718594    oracle     640        226
0xcd9652f1 4751363    oracle     640        226
0xcd9652f2 4784132    oracle     640        226
0xcd9652f3 4816901    oracle     640        226
0xcd9652f4 4849670    oracle     640        226

The recommended kernel.sem value from Oracle is "250 32000 100 128". I 
have tried to reduce the maximum semaphores per array (1st value) while 
increasing the max number of arrays. That tends to reduce the ipc_lock 
contention in kernel, but it is against Oracle's recommendation.

-Longman

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/