linux-kernel - Re: [RFC PATCH v3 13/15] context_tracking,x86: Add infrastructure to defer kernel TLBI

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <xhsmhv7vr63vj.mognet@vschneid-thinkpadt14sgen2i.remote.csb>
Date: Tue, 10 Dec 2024 14:53:36 +0100
From: Valentin Schneider <vschneid@...hat.com>
To: Petr Tesarik <ptesarik@...e.com>, Peter Zijlstra <peterz@...radead.org>
Cc: Dave Hansen <dave.hansen@...el.com>, linux-kernel@...r.kernel.org,
 linux-doc@...r.kernel.org, kvm@...r.kernel.org, linux-mm@...ck.org,
 bpf@...r.kernel.org, x86@...nel.org, rcu@...r.kernel.org,
 linux-kselftest@...r.kernel.org, Steven Rostedt <rostedt@...dmis.org>,
 Masami Hiramatsu <mhiramat@...nel.org>, Jonathan Corbet <corbet@....net>,
 Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>,
 Borislav Petkov <bp@...en8.de>, Dave Hansen <dave.hansen@...ux.intel.com>,
 "H. Peter Anvin" <hpa@...or.com>, Paolo Bonzini <pbonzini@...hat.com>,
 Wanpeng Li <wanpengli@...cent.com>, Vitaly Kuznetsov
 <vkuznets@...hat.com>, Andy Lutomirski <luto@...nel.org>, Frederic
 Weisbecker <frederic@...nel.org>, "Paul E. McKenney" <paulmck@...nel.org>,
 Neeraj
 Upadhyay <quic_neeraju@...cinc.com>, Joel Fernandes
 <joel@...lfernandes.org>, Josh Triplett <josh@...htriplett.org>, Boqun
 Feng <boqun.feng@...il.com>, Mathieu Desnoyers
 <mathieu.desnoyers@...icios.com>, Lai Jiangshan <jiangshanlai@...il.com>,
 Zqiang <qiang.zhang1211@...il.com>, Andrew Morton
 <akpm@...ux-foundation.org>, Uladzislau Rezki <urezki@...il.com>,
 Christoph Hellwig <hch@...radead.org>, Lorenzo Stoakes
 <lstoakes@...il.com>, Josh Poimboeuf <jpoimboe@...nel.org>, Jason Baron
 <jbaron@...mai.com>, Kees Cook <keescook@...omium.org>, Sami Tolvanen
 <samitolvanen@...gle.com>, Ard Biesheuvel <ardb@...nel.org>, Nicholas
 Piggin <npiggin@...il.com>, Juerg Haefliger
 <juerg.haefliger@...onical.com>, Nicolas Saenz Julienne
 <nsaenz@...nel.org>, "Kirill A. Shutemov"
 <kirill.shutemov@...ux.intel.com>, Nadav Amit <namit@...are.com>, Dan
 Carpenter <error27@...il.com>, Chuang Wang <nashuiliang@...il.com>, Yang
 Jihong <yangjihong1@...wei.com>, Petr Mladek <pmladek@...e.com>, "Jason A.
 Donenfeld" <Jason@...c4.com>, Song Liu <song@...nel.org>, Julian Pidancet
 <julian.pidancet@...cle.com>, Tom Lendacky <thomas.lendacky@....com>,
 Dionna Glaze <dionnaglaze@...gle.com>, Thomas Weißschuh
 <linux@...ssschuh.net>, Juri Lelli <juri.lelli@...hat.com>, Marcelo
 Tosatti <mtosatti@...hat.com>, Yair Podemsky <ypodemsk@...hat.com>, Daniel
 Wagner <dwagner@...e.de>
Subject: Re: [RFC PATCH v3 13/15] context_tracking,x86: Add infrastructure
 to defer kernel TLBI

On 09/12/24 15:42, Petr Tesarik wrote:
> On Mon, 9 Dec 2024 13:12:49 +0100
> Peter Zijlstra <peterz@...radead.org> wrote:
>
>> On Mon, Dec 09, 2024 at 01:04:43PM +0100, Valentin Schneider wrote:
>>
>> > > But I wonder what exactly was the original scenario encountered by
>> > > Valentin. I mean, if TLB entry invalidations were necessary to sync
>> > > changes to kernel text after flipping a static branch, then it might be
>> > > less overhead to make a list of affected pages and call INVLPG on them.
>>
>> No; TLB is not involved with text patching (on x86).
>>
>> > > Valentin, do you happen to know?
>> >
>> > So from my experimentation (hackbench + kernel compilation on housekeeping
>> > CPUs, dummy while(1) userspace loop on isolated CPUs), the TLB flushes only
>> > occurred from vunmap() - mainly from all the hackbench threads coming and
>> > going.
>>
>> Right, we have virtually mapped stacks.
>
> Wait... Are you talking about the kernel stac? But that's only 4 pages
> (or 8 pages with KASAN), so that should be easily handled with INVLPG.
> No CR4 dances are needed for that.
>
> What am I missing?
>

So the gist of the IPI deferral thing is to coalesce IPI callbacks into a
single flag value that is read & acted on upon kernel entry. Freeing a
task's kernel stack is not the only thing that can issue a vunmap(), so
instead of tracking all the pages affected by the unmap (which is
potentially an ever-growing memory leak as long as no kernel entry happens
on the isolated CPUs), we just flush everything.

Quick tracing with my dummy benchmark mostly shows

  vfree_atomic() -> drain_vmap_work()

but pretty much any vfree() / kvfree_rcu() from the housekeeping CPUs can
get us that IPI.