linux-kernel - Re: [patch 53/60] x86/mm: Use/Fix PCID to optimize user/kernel switches

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20171205220557.GX3165@worktop.lehotels.local>
Date:   Tue, 5 Dec 2017 23:05:57 +0100
From:   Peter Zijlstra <peterz@...radead.org>
To:     Andy Lutomirski <luto@...nel.org>
Cc:     Thomas Gleixner <tglx@...utronix.de>,
        LKML <linux-kernel@...r.kernel.org>, X86 ML <x86@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Dave Hansen <dave.hansen@...el.com>,
        Borislav Petkov <bpetkov@...e.de>,
        Greg KH <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...gle.com>,
        Hugh Dickins <hughd@...gle.com>,
        Brian Gerst <brgerst@...il.com>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Denys Vlasenko <dvlasenk@...hat.com>,
        Rik van Riel <riel@...hat.com>,
        Boris Ostrovsky <boris.ostrovsky@...cle.com>,
        Juergen Gross <jgross@...e.com>,
        David Laight <David.Laight@...lab.com>,
        Eduardo Valentin <eduval@...zon.com>, aliguori@...zon.com,
        Will Deacon <will.deacon@....com>,
        Daniel Gruss <daniel.gruss@...k.tugraz.at>
Subject: Re: [patch 53/60] x86/mm: Use/Fix PCID to optimize user/kernel
 switches

On Tue, Dec 05, 2017 at 01:46:36PM -0800, Andy Lutomirski wrote:
> On Mon, Dec 4, 2017 at 6:07 AM, Thomas Gleixner <tglx@...utronix.de> wrote:
> > We can use PCID to retain the TLBs across CR3 switches; including
> > those now part of the user/kernel switch. This increases performance
> > of kernel entry/exit at the cost of more expensive/complicated TLB
> > flushing.
> >
> > Now that we have two address spaces, one for kernel and one for user
> > space, we need two PCIDs per mm. We use the top PCID bit to indicate a
> > user PCID (just like we use the PFN LSB for the PGD). Since we do TLB
> > invalidation from kernel space, the existing code will only invalidate
> > the kernel PCID, we augment that by marking the corresponding user
> > PCID invalid, and upon switching back to userspace, use a flushing CR3
> > write for the switch.
> >
> > In order to access the user_pcid_flush_mask we use PER_CPU storage,
> > which means the previously established SWAPGS vs CR3 ordering is now
> > mandatory and required.
> >
> > Having to do this memory access does require additional registers,
> > most sites have a functioning stack and we can spill one (RAX), sites
> > without functional stack need to otherwise provide the second scratch
> > register.
> >
> > Note: PCID is generally available on Intel Sandybridge and later CPUs.
> > Note: Up until this point TLB flushing was broken in this series.
> 
> I haven't checked that hard which patch introduces this bug, but it
> seems that, with this applied, nothing propagates
> non-mm-switch-related flushes to usermode.  Shouldn't
> flush_tlb_func_common() contain a call to invalidate_user_asid() near
> the bottom?  Alternatively, it could be in local_flush_tlb() and
> __flush_tlb_single() (or whatever the hell the flush-one-usermode-TLB
> function ends up being called).

__native_flush_tlb_single() has the invalidate_user_asid()
__native_flush_tlb() has the invalidate_user_asid().

Which should be exactly that last option you mention.

> Also, on a somewhat related note, __flush_tlb_single() is called from
> both flush_tlb_func_common() and do_kernel_range_flush.  That sounds
> wrong.

Fixed that in the patches I send out earlier today.