linux-kernel - Re: [PATCH v17 02/16] preempt: Track NMI nesting to separate per-CPU counter

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20260206084505.GP1282955@noisy.programming.kicks-ass.net>
Date: Fri, 6 Feb 2026 09:45:05 +0100
From: Peter Zijlstra <peterz@...radead.org>
To: Boqun Feng <boqun@...nel.org>
Cc: Lyude Paul <lyude@...hat.com>, rust-for-linux@...r.kernel.org,
	linux-kernel@...r.kernel.org, Thomas Gleixner <tglx@...utronix.de>,
	Boqun Feng <boqun.feng@...il.com>,
	Daniel Almeida <daniel.almeida@...labora.com>,
	Miguel Ojeda <ojeda@...nel.org>,
	Alex Gaynor <alex.gaynor@...il.com>, Gary Guo <gary@...yguo.net>,
	Björn Roy Baron <bjorn3_gh@...tonmail.com>,
	Benno Lossin <lossin@...nel.org>,
	Andreas Hindborg <a.hindborg@...nel.org>,
	Alice Ryhl <aliceryhl@...gle.com>, Trevor Gross <tmgross@...ch.edu>,
	Danilo Krummrich <dakr@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Ingo Molnar <mingo@...hat.com>, Will Deacon <will@...nel.org>,
	Waiman Long <longman@...hat.com>,
	Joel Fernandes <joelagnelf@...dia.com>
Subject: Re: [PATCH v17 02/16] preempt: Track NMI nesting to separate per-CPU
 counter

On Thu, Feb 05, 2026 at 02:07:40PM -0800, Boqun Feng wrote:
> On Wed, Feb 04, 2026 at 12:12:34PM +0100, Peter Zijlstra wrote:
> [...]
> >  DEFINE_PER_CPU_CACHE_HOT(unsigned long, cpu_current_top_of_stack) = TOP_OF_INIT_STACK;
> > diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h
> > index d57cab4d4c06..77defd9624bf 100644
> > --- a/include/linux/hardirq.h
> > +++ b/include/linux/hardirq.h
> > @@ -108,15 +108,14 @@ void irq_exit_rcu(void);
> >  	do {							\
> >  		lockdep_off();					\
> >  		arch_nmi_enter();				\
> > -		BUG_ON(in_nmi() == NMI_MASK);			\
> > -		__preempt_count_add(NMI_OFFSET + HARDIRQ_OFFSET);	\
> > +		__preempt_count_nmi_enter();			\
> >  	} while (0)
> >  
> >  #define nmi_enter()						\
> >  	do {							\
> >  		__nmi_enter();					\
> >  		lockdep_hardirq_enter();			\
> > -		ct_nmi_enter();				\
> > +		ct_nmi_enter();					\
> >  		instrumentation_begin();			\
> >  		ftrace_nmi_enter();				\
> >  		instrumentation_end();				\
> > @@ -125,7 +124,7 @@ void irq_exit_rcu(void);
> >  #define __nmi_exit()						\
> >  	do {							\
> >  		BUG_ON(!in_nmi());				\
> > -		__preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET);	\
> > +		__preempt_count_nmi_exit();			\
> >  		arch_nmi_exit();				\
> >  		lockdep_on();					\
> >  	} while (0)
> > diff --git a/include/linux/preempt.h b/include/linux/preempt.h
> > index d964f965c8ff..7617ca97f442 100644
> > --- a/include/linux/preempt.h
> > +++ b/include/linux/preempt.h
> [...]
> > @@ -132,6 +145,27 @@ static __always_inline unsigned char interrupt_context_level(void)
> >  # define in_task()		(!(preempt_count() & (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET)))
> >  #endif
> >  
> > +#ifndef CONFIG_PREEMPT_LONG
> > +DECLARE_PER_CPU(unsigned int, nmi_nesting);
> > +
> > +#define __preempt_count_nmi_enter()				\
> > +	do {							\
> > +		unsigned int _o = NMI_MASK + HARDIRQ_OFFSET;	\
> > +		__this_cpu_inc(nmi_nesting);			\
> > +		_o -= (preempt_count() & NMI_MASK);		\
> > +		__preempt_count_add(_o);			\
> > +	} while (0)
> > +
> > +#define __preempt_count_nmi_exit()				\
> > +	do {							\
> > +		unsigned int _o = HARDIRQ_OFFSET;		\
> > +		if (!__this_cpu_dec_return(nmi_nesting))	\
> > +			_o += NMI_MASK;				\
> > +		__preempt_count_sub(_o);			\
> > +	} while (0)
> > +
> > +#endif
> > +
> 
> We need to move it into include/linux/hardirq.h because percpu is not
> included in <linux/preempt.h>.

That is fine. I also realized you can move the variants from
arch/x86/asm/preempt.h right next to it, it only depends on
PREEMPT_LONG, not anything else, so there is nothing arch specific to
it.

Avoids that getting duplicated on arm64,s390 etc.