linux-kernel - Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 29 Aug 2016 16:55:02 -0700
From:   Andy Lutomirski <luto@...capital.net>
To:     Rik van Riel <riel@...hat.com>
Cc:     Thomas Gleixner <tglx@...utronix.de>,
        Benjamin Serebrin <serebrin@...gle.com>,
        Borislav Petkov <bp@...e.de>, Ingo Molnar <mingo@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Mel Gorman <mgorman@...e.de>, "H. Peter Anvin" <hpa@...or.com>,
        Peter Zijlstra <peterz@...radead.org>
Subject: Re: [PATCH RFC UGLY] x86,mm,sched: make lazy TLB mode even lazier

On Aug 29, 2016 7:54 AM, "Rik van Riel" <riel@...hat.com> wrote:
>
> On Sun, 2016-08-28 at 01:11 -0700, Andy Lutomirski wrote:
> > On Aug 25, 2016 9:06 PM, "Rik van Riel" <riel@...hat.com> wrote:
> > >
> > > Subject: x86,mm,sched: make lazy TLB mode even lazier
> > >
> > > Lazy TLB mode can result in an idle CPU being woken up for a TLB
> > > flush, when all it really needed to do was flush %cr3 before the
> > > next context switch.
> > >
> > > This is mostly fine on bare metal, though sub-optimal from a power
> > > saving point of view, and deeper C states could make TLB flushes
> > > take a little longer than desired.
> > >
> > > On virtual machines, the pain can be much worse, especially if a
> > > currently non-running VCPU is woken up for a TLB invalidation
> > > IPI, on a CPU that is busy running another task. It could take
> > > a while before that IPI is handled, leading to performance issues.
> > >
> > > This patch is still ugly, and the sched.h include needs to be
> > > cleaned
> > > up a lot (how would the scheduler people like to see the context
> > > switch
> > > blocking abstracted?)
> > >
> > > This patch deals with the issue by introducing a third tlb state,
> > > TLBSTATE_FLUSH, which causes %cr3 to be flushed at the next
> > > context switch. A CPU is transitioned from TLBSTATE_LAZY to
> > > TLBSTATE_FLUSH with the rq lock held, to prevent context switches.
> > >
> > > Nothing is done for a CPU that is already in TLBSTATE_FLUH mode.
> > >
> > > This patch is totally untested, because I am at a conference right
> > > now, and Benjamin has the test case :)
> > >
> >
> > I haven't had a chance to seriously read the code yet, but what
> > happens when the mm is deleted outright?  Or is the idea that a
> > reference is held until all the lazy users are gone, too?
>
> Worst case we send a TLB flush to a CPU that does
> not need it.
>
> As not sending an IPI will be faster than sending
> one, I do not think the tradeoff will be much
> different for a system with PCID.

If we were fully non-lazy, we wouldn't need to send these IPIs at all,
right?  We would just keep cr3 pointing at swapper_pg_dir when not
actively using the mm.  The problem with doing that without PCID is
that cr3 writes are really slow.  Or am I missing something?