linux-kernel - Re: [PATCH] x86: serialize LVTT and TSC

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150805162535.GA2461245@devbig257.prn2.facebook.com>
Date:	Wed, 5 Aug 2015 09:25:46 -0700
From:	Shaohua Li <shli@...com>
To:	Ingo Molnar <mingo@...nel.org>
CC:	Thomas Gleixner <tglx@...utronix.de>, <x86@...nel.org>,
	<linux-kernel@...r.kernel.org>, <Kernel-team@...com>,
	Suresh Siddha <suresh.b.siddha@...el.com>,
	"H. Peter Anvin" <hpa@...or.com>, <stable@...r.kernel.org>,
	<ak@...ux.intel.com>, <lenb@...nel.org>, <fenghua.yu@...el.com>
Subject: Re: [PATCH] x86: serialize LVTT and TSC_DEADLINE write

On Wed, Aug 05, 2015 at 10:44:24AM +0200, Ingo Molnar wrote:
> 
> * Shaohua Li <shli@...com> wrote:
> 
> > On Sun, Aug 02, 2015 at 09:41:08PM +0200, Thomas Gleixner wrote:
> > > On Sun, 2 Aug 2015, Shaohua Li wrote:
> > > 
> > > > On Sat, Aug 01, 2015 at 12:10:41PM +0200, Thomas Gleixner wrote:
> > > > > On Fri, 31 Jul 2015, Shaohua Li wrote:
> > > > > > @@ -336,6 +336,22 @@ static void __setup_APIC_LVTT(unsigned int clocks, int oneshot, int irqen)
> > > > > >  	apic_write(APIC_LVTT, lvtt_value);
> > > > > >  
> > > > > >  	if (lvtt_value & APIC_LVT_TIMER_TSCDEADLINE) {
> > > > > > +		u64 msr;
> > > > > > +
> > > > > > +		/*
> > > > > > +		 * See Intel SDM: TSC-Deadline Mode chapter. In xAPIC mode,
> > > > > > +		 * writing APIC LVTT and TSC_DEADLINE MSR isn't serialized.
> > > > > > +		 * This uses the algorithm described in Intel SDM to serialize
> > > > > > +		 * the two writes
> > > > > > +		 * */
> > > > > > +		while (1) {
> > > > > > +			wrmsrl(MSR_IA32_TSC_DEADLINE, -1L);
> > > > > > +			rdmsrl(MSR_IA32_TSC_DEADLINE, msr);
> > > > > > +			if (msr)
> > > > > > +				break;
> > > > > > +		}
> > > > > > +		wrmsrl(MSR_IA32_TSC_DEADLINE, 0);
> > > > > 
> > > > > 
> > > > > I think this is exceptionally silly. A proper fence after the
> > > > > apic_write() should have the same effect.
> > > > 
> > > > Not sure what happens in the hardware, I could have a try of fence, but
> > > > I'd prefer using the algorithm Intel described. This is not a fast path,
> > > 
> > > s/algorithm/voodoo/
> > > 
> > > > the loop will exit immediately regardless the issue occurs anyway.
> > > 
> > > Well, the SDM also says:
> > > 
> > >  "To allow for efficient access to the APIC registers in x2APIC mode,
> > >   the serializing semantics of WRMSR are relaxed when writing to the
> > >   APIC registers. Thus, system software should not use “WRMSR to APIC
> > >   registers in x2APIC mode” as a serializing instruction. Read and write
> > >   accesses to the APIC registers will occur in program order. A WRMSR to
> > >   an APIC register may complete before all preceding stores are globally
> > >   visible; software can prevent this by inserting a serializing
> > >   instruction, an SFENCE, or an MFENCE before the WRMSR."
> > > 
> > > And that's what happens here. The write to the LVT has not yet hit the
> > > APIC, so the WRMSR has no effect.
> > 
> > What you quoted is for x2APIC, I didn't see similar description for
> > xAPIC.
> > 
> > Tested mfence here, it does work. But I'm not convinced it's the right thing. 
> > the xAPIC access is memory mapped IO, mfence is nothing related to it. [...]
> 
> Huh?
> 
> Can you cite the SDM that says that MFENCE will have no effect on instructions 
> that deal with accesses that are not to system RAM (i.e. are memory mapped IO)?

Hmm, I didn't mean mfence can't serialize the instructions. For a true
IO, a serialization can't guarantee device finishes the IO, we generally
read some safe IO registers to wait IO finish. I completely don't know
if this case fits here though.

Thanks,
Shaohua
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/