linux-kernel - Re: NOHZ tick-stop error: Non-RCU local softirq work is pending

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20201211032425.GC2657@paulmck-ThinkPad-P72>
Date:   Thu, 10 Dec 2020 19:24:25 -0800
From:   "Paul E. McKenney" <paulmck@...nel.org>
To:     Frederic Weisbecker <frederic@...nel.org>
Cc:     linux-kernel@...r.kernel.org
Subject: Re: NOHZ tick-stop error: Non-RCU local softirq work is pending

On Fri, Dec 11, 2020 at 02:40:31AM +0100, Frederic Weisbecker wrote:
> On Thu, Dec 10, 2020 at 04:46:38PM -0800, Paul E. McKenney wrote:
> > > diff --git a/kernel/softirq.c b/kernel/softirq.c
> > > index 09229ad82209..7d558cb7a037 100644
> > > --- a/kernel/softirq.c
> > > +++ b/kernel/softirq.c
> > > @@ -650,7 +650,9 @@ static void run_ksoftirqd(unsigned int cpu)
> > >  		 * We can safely run softirq on inline stack, as we are not deep
> > >  		 * in the task stack here.
> > >  		 */
> > > -		__do_softirq();
> > > +		do {
> > > +			__do_softirq();
> > > +		} while (kthread_should_park() && local_softirq_pending());
> > >  		local_irq_enable();
> > >  		cond_resched();
> > >  		return;
> > 
> > Huh.  I guess that self-propagating timers, RCU callbacks, and the
> > like are non-problems because they cannot retrigger while interrupts
> > are disabled?  But can these things reappear just after the
> > local_irq_enable()?
> 
> Exactly, unless those things have been disabled in CPU hotplug callbacks
> before smpboot_park_threads() (and we are not too deep in CPU hotplug at that
> stage, cpu_online() is still true), they should be able to raise again
> the softirqs. Yeah I'm pretty sure the above won't be enough.
> 
> > In the case of RCU, softirq would need to run on this CPU, which it won't,
> > so we are good in that case.  (Any stranded callbacks will be requeued
> > onto some other CPU later in the CPU-hotplug offline processing.)
> 
> Ah that's good, so at least we shouldn't worry about pending RCU softirqs
> after ksoftirqd has parked. But I also see at least SCHED_SOFTIRQ and
> TIMER_SOFTIRQ in your list.
> 
> Perhaps we should have some flush_softirq() somewhere late in cpu hotplug,
> assuming this is actually necessary and none of these will requeue themselves...

Indeed, let's see if we have a problem before we worry too much about
fixing it.  ;-)

I still get the "NOHZ tick-stop error", but more than three times less
often, so definite improvement.  One of the console.log file is attached.

							Thanx, Paul

Download attachment "console1.log.gz" of type "application/gzip" (804216 bytes)