linux-kernel - Re: [RFC PATCH] qspinlock: Improve performance by reducing load instruction rollback

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAOGi=dNiTz3SxmQRoAN8ZrQGaT9tOg-12LgH5Xq6kAtQcRdn8g@mail.gmail.com>
Date:	Wed, 21 Oct 2015 13:30:36 +0800
From:	Ling Ma <ling.ma.program@...il.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	mingo@...hat.com, linux-kernel@...r.kernel.org,
	Ma Ling <ling.ml@...baba-inc.com>,
	Waiman Long <Waiman.Long@....com>
Subject: Re: [RFC PATCH] qspinlock: Improve performance by reducing load
 instruction rollback

2015-10-20 17:16 GMT+08:00 Peter Zijlstra <peterz@...radead.org>:
> On Tue, Oct 20, 2015 at 11:24:02AM +0800, Ling Ma wrote:
>> 2015-10-19 17:46 GMT+08:00 Peter Zijlstra <peterz@...radead.org>:
>> > On Mon, Oct 19, 2015 at 10:27:22AM +0800, ling.ma.program@...il.com wrote:
>> >> From: Ma Ling <ling.ml@...baba-inc.com>
>> >>
>> >> All load instructions can run speculatively but they have to follow
>> >> memory order rule in multiple cores as below:
>> >> _x = _y = 0
>> >>
>> >> Processor 0                           Processor 1
>> >>
>> >> mov r1, [ _y]  //M1                   mov [ _x], 1  //M3
>> >> mov r2, [ _x]  //M2                   mov [ _y], 1  //M4
>> >>
>> >> If r1 = 1, r2 must be 1
>> >>
>> >> In order to guarantee above rule, although Processor 0 execute
>> >> M1 and M2 instruction out of order, they are kept in ROB,
>> >> when load buffer for _x in Processor 0 received the update
>> >> message from Processor 1, Processor 0 need to roll back
>> >> from M2 instruction, which will flush the whole pipeline,
>> >> the latency is over the penalty from branch prediction miss.
>> >>
>> >> In this patch we use lock cmpxchg instruction to force load
>> >
>> > "lock cmpxchg" makes me think you're working on x86.
>> >
>> >> instructions to be serialization,
>> >
>> > smp_rmb() does that, and that's 'free' on x86. Because x86 doesn't do
>> > read reordering.
>> >
>> >> the destination operand
>> >> receives a write cycle without regard to the result of
>> >> the comparison, which can help us to reduce the penalty
>> >> from load instruction roll back.
>> >
>> > And that makes me think I'm not understanding what you're getting at. If
>> > you need to force memory order, a "fence" (or smp_mb()) would still be
>> > cheaper than endlessly pulling the line into exclusive state for no
>> > reason, right?
>>
>> Peter,
>>
>> we tested instruction lfence, but we hard to see any benefit, lfence
>> only force load instruction ,
>> but load instruction still will rollback ,actually cmpxchg behavior is
>> more like write operation,
>> so we choose it.
>
> But why? I'm just not getting this.
>
> Also LOCK CMPXCHG is 24 cycles when hot, that's almost as bad as
> a pipeline flush, and it can be many times worse when it needs to
> actually fetch memory from further than L1.

We will clarify the root cause about this question soon.

Thanks
Ling
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/