linux-kernel - Re: x86 memory barrier: why does Linux prefer MFENCE to Locked ADD?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160803125025.GB680@khazad-dum.debian.net>
Date:	Wed, 3 Aug 2016 09:50:25 -0300
From:	Henrique de Moraes Holschuh <hmh@....eng.br>
To:	"Michael S. Tsirkin" <mst@...hat.com>
Cc:	"H. Peter Anvin" <hpa@...or.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...nel.org>,
	Dexuan Cui <decui@...rosoft.com>,
	"linux-x86_64@...r.kernel.org" <linux-x86_64@...r.kernel.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>,
	David Howells <dhowells@...hat.com>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: x86 memory barrier: why does Linux prefer MFENCE to Locked ADD?

On Wed, 03 Aug 2016, Michael S. Tsirkin wrote:
> > And I'm still discussing this with the hardware people.  It seems we
> > can do this for *most* things, but not all; the question is where
> > exactly we need to do something different.

Let's hope the "hardware guys" get back to you soon :(


     HSD162/BDM116  MOVNTDQA From WC Memory May Pass Earlier Locked
                    Instructions

     Problem: An execution of (V)MOVNTDQA (streaming load instruction)
     that loads from WC (write combining) memory may appear to pass an
     earlier locked instruction that accesses a different cache line.

     Implication: Software that expects a lock to fence subsequent
     (V)MOVNTDQA instructions may not operate properly.

     Workaround: None identified.  Software that relies on a locked
     instruction to fence subsequent executions of (V)MOVNTDQA should
     insert an MFENCE instruction between the locked instruction and
     subsequent (V)MOVNTDQA instruction.



     SKL079   MOVNTDQA From WC Memory May Pass Earlier MFENCE Instructions

     Problem: An execution of MOVNTDQA or VMOVNTDQA that loads from WC
     (write combining) memory may appear to pass an earlier execution of
     the MFENCE instruction.

     Implication: When this erratum occurs, an execution of MOVNTDQA or
     VMOVNTDQA may appear to execute before memory operations that
     precede the earlier MFENCE instruction.  Software that uses MFENCE
     to order subsequent executions of the MOVNTDQA instructions may not
     operate properly.

     Workaround: It is possible for the BIOS to contain a workaround for
     this erratum.  For the steppings affected, see the Summary Table of
     Changes.


These are just examples.  Intel might have other errata related to
*FENCE or LOCK, and AMD might have its share of model-specific LOCK or
*FENCE oddities as well (I didn't check).

Note that Skylake is broken in exactly the opposite way that Haswell and
Broadwell are.  Fortunately, Skylake could be fixed through a microcode
update, but still...

The point is that we indeed need to be careful if we want to switch away
from *FENCE.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh