linux-kernel - Re: [PATCH 0/5] x86,smp: make ticket spinlock proportional backoff w/ auto tuning

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <50F2F7C5.6070708@linux.vnet.ibm.com>
Date:	Sun, 13 Jan 2013 23:37:01 +0530
From:	Raghavendra K T <raghavendra.kt@...ux.vnet.ibm.com>
To:	Rik van Riel <riel@...hat.com>
CC:	Rafael Aquini <aquini@...hat.com>, linux-kernel@...r.kernel.org,
	walken@...gle.com, eric.dumazet@...il.com, lwoodman@...hat.com,
	jeremy@...p.org, Jan Beulich <JBeulich@...ell.com>,
	knoel@...hat.com, chegu_vinod@...com, mingo@...hat.com
Subject: Re: [PATCH 0/5] x86,smp: make ticket spinlock proportional backoff
 w/ auto tuning

On 01/12/2013 01:41 AM, Rik van Riel wrote:
> On 01/10/2013 12:36 PM, Raghavendra K T wrote:
>> * Rafael Aquini <aquini@...hat.com> [2013-01-10 00:27:23]:
>>
>>> On Wed, Jan 09, 2013 at 06:20:35PM +0530, Raghavendra K T wrote:
>>>> I ran kernbench on 32 core (mx3850) machine with 3.8-rc2 base.
>>>> x base_3.8rc2
>>>> + rik_backoff
>>>>      N           Min           Max        Median
>>>> Avg        Stddev
>>>> x   8       222.977        231.16       227.735       227.388
>>>> 3.1512986
>>>> +   8        218.75       232.347      229.1035     228.25425
>>>> 4.2730225
>>>> No difference proven at 95.0% confidence
>>>
>>> I got similar results on smaller systems (1 socket, dual-cores and
>>> quad-cores)
>>> when running Rik's latest series, no big difference for good nor for
>>> worse,
>>> but I also think Rik's work is meant to address bigger systems with
>>> more cores
>>> contending for any given spinlock.
>>
>> I was able to do the test on same 32 core machine with
>> 4 guests (8GB RAM, 32 vcpu).
>> Here are the results
>>
>> base = 3.8-rc2
>> patched =  base + Rik V3 backoff series [patch 1-4]
>
> I believe I understand why this is happening.
>
> Modern Intel and AMD CPUs have a feature called Pause Loop Exiting (PLE)
> and Pause Filter (PF), respectively.  This feature is used to trap to
> the host when the guest is spinning on a spinlock.
>
> This allows the host to run something else, and having the spinner
> temporarily yield the CPU.  Effectively, this causes the KVM code
> to already do some limited amount of spinlock backoff code, in the
> host.
>
> Adding more backoff code in the guest can lead to wild delays in
> acquiring locks, and generally bad performance.

Yes agree with you.

> I suspect that when running in a virtual machine, we should limit
> the delay factor to something much smaller, since the host will take
> care of most of the backoff for us.
>

Even for non-PLE case I believe it would be difficult to tune delay,
because of VCPU scheduling and LHP.

> Maybe a maximum delay value of ~10 would do the trick for KVM
> guests.
>
> We should be able to get this right by placing the value for the
> maximum delay in a __read_mostly section and setting it to something
> small from an init function when we detect we are running in a
> virtual machine.
>
> Let me cook up, and test, a patch that does that...

Sure.. Awaiting and happy to test the patches.
I also tried few things on my own and also how it behaves without patch
4. Nothing helped.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/