linux-kernel - Re: [patch 53/60] x86/mm: Use/Fix PCID to optimize user/kernel switches

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 5 Dec 2017 13:46:36 -0800
From:   Andy Lutomirski <luto@...nel.org>
To:     Thomas Gleixner <tglx@...utronix.de>
Cc:     LKML <linux-kernel@...r.kernel.org>, X86 ML <x86@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andy Lutomirsky <luto@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Dave Hansen <dave.hansen@...el.com>,
        Borislav Petkov <bpetkov@...e.de>,
        Greg KH <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...gle.com>,
        Hugh Dickins <hughd@...gle.com>,
        Brian Gerst <brgerst@...il.com>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Denys Vlasenko <dvlasenk@...hat.com>,
        Rik van Riel <riel@...hat.com>,
        Boris Ostrovsky <boris.ostrovsky@...cle.com>,
        Juergen Gross <jgross@...e.com>,
        David Laight <David.Laight@...lab.com>,
        Eduardo Valentin <eduval@...zon.com>, aliguori@...zon.com,
        Will Deacon <will.deacon@....com>,
        Daniel Gruss <daniel.gruss@...k.tugraz.at>
Subject: Re: [patch 53/60] x86/mm: Use/Fix PCID to optimize user/kernel switches

On Mon, Dec 4, 2017 at 6:07 AM, Thomas Gleixner <tglx@...utronix.de> wrote:
> We can use PCID to retain the TLBs across CR3 switches; including
> those now part of the user/kernel switch. This increases performance
> of kernel entry/exit at the cost of more expensive/complicated TLB
> flushing.
>
> Now that we have two address spaces, one for kernel and one for user
> space, we need two PCIDs per mm. We use the top PCID bit to indicate a
> user PCID (just like we use the PFN LSB for the PGD). Since we do TLB
> invalidation from kernel space, the existing code will only invalidate
> the kernel PCID, we augment that by marking the corresponding user
> PCID invalid, and upon switching back to userspace, use a flushing CR3
> write for the switch.
>
> In order to access the user_pcid_flush_mask we use PER_CPU storage,
> which means the previously established SWAPGS vs CR3 ordering is now
> mandatory and required.
>
> Having to do this memory access does require additional registers,
> most sites have a functioning stack and we can spill one (RAX), sites
> without functional stack need to otherwise provide the second scratch
> register.
>
> Note: PCID is generally available on Intel Sandybridge and later CPUs.
> Note: Up until this point TLB flushing was broken in this series.

I haven't checked that hard which patch introduces this bug, but it
seems that, with this applied, nothing propagates
non-mm-switch-related flushes to usermode.  Shouldn't
flush_tlb_func_common() contain a call to invalidate_user_asid() near
the bottom?  Alternatively, it could be in local_flush_tlb() and
__flush_tlb_single() (or whatever the hell the flush-one-usermode-TLB
function ends up being called).

Also, on a somewhat related note, __flush_tlb_single() is called from
both flush_tlb_func_common() and do_kernel_range_flush.  That sounds
wrong.