linux-kernel - Re: [PATCH RFC tip/core/rcu 14/14] rcu/nohz: Make multi_cpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAEXW_YRm6cBp-J3tPeRH+Jx=38LaAtNoCM9f0FeFG+ZBZKBH6w@mail.gmail.com>
Date:   Mon, 12 Aug 2019 21:33:51 -0400
From:   Joel Fernandes <joel@...lfernandes.org>
To:     "Paul E. McKenney" <paulmck@...ux.ibm.com>
Cc:     Frederic Weisbecker <frederic@...nel.org>,
        rcu <rcu@...r.kernel.org>, LKML <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...nel.org>,
        Lai Jiangshan <jiangshanlai@...il.com>,
        dipankar <dipankar@...ibm.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        Josh Triplett <josh@...htriplett.org>,
        Thomas Glexiner <tglx@...utronix.de>,
        Peter Zilstra <peterz@...radead.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        David Howells <dhowells@...hat.com>,
        Eric Dumazet <edumazet@...gle.com>,
        Frederic Weisbecker <fweisbec@...il.com>,
        Oleg Nesterov <oleg@...hat.com>
Subject: Re: [PATCH RFC tip/core/rcu 14/14] rcu/nohz: Make multi_cpu_stop()
 enable tick on all online CPUs

On Mon, Aug 12, 2019 at 7:23 PM Paul E. McKenney <paulmck@...ux.ibm.com> wrote:
>
> On Mon, Aug 12, 2019 at 11:02:33PM +0200, Frederic Weisbecker wrote:
> > On Fri, Aug 02, 2019 at 08:15:01AM -0700, Paul E. McKenney wrote:
> > > The multi_cpu_stop() function relies on the scheduler to gain control from
> > > whatever is running on the various online CPUs, including any nohz_full
> > > CPUs running long loops in kernel-mode code.  Lack of the scheduler-clock
> > > interrupt on such CPUs can delay multi_cpu_stop() for several minutes
> > > and can also result in RCU CPU stall warnings.  This commit therefore
> > > causes multi_cpu_stop() to enable the scheduler-clock interrupt on all
> > > online CPUs.
> > >
> > > Signed-off-by: Paul E. McKenney <paulmck@...ux.ibm.com>
> > > ---
> > >  kernel/stop_machine.c | 9 ++++++++-
> > >  1 file changed, 8 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
> > > index b4f83f7bdf86..a2659f61ed92 100644
> > > --- a/kernel/stop_machine.c
> > > +++ b/kernel/stop_machine.c
> > > @@ -20,6 +20,7 @@
> > >  #include <linux/smpboot.h>
> > >  #include <linux/atomic.h>
> > >  #include <linux/nmi.h>
> > > +#include <linux/tick.h>
> > >  #include <linux/sched/wake_q.h>
> > >
> > >  /*
> > > @@ -187,15 +188,19 @@ static int multi_cpu_stop(void *data)
> > >  {
> > >     struct multi_stop_data *msdata = data;
> > >     enum multi_stop_state curstate = MULTI_STOP_NONE;
> > > -   int cpu = smp_processor_id(), err = 0;
> > > +   int cpu, err = 0;
> > >     const struct cpumask *cpumask;
> > >     unsigned long flags;
> > >     bool is_active;
> > >
> > > +   for_each_online_cpu(cpu)
> > > +           tick_nohz_dep_set_cpu(cpu, TICK_DEP_MASK_RCU);
> >
> > Looks like it's not the right fix but, should you ever need to set an
> > all-CPUs (system wide) tick dependency in the future, you can use tick_set_dep().
>
> Indeed, I have dropped this patch, but I now do something similar in
> RCU's CPU-hotplug notifiers.  Which does have an effect, especially on
> the system that isn't subject to the insane-latency cpu_relax().
>
> Plus I am having to put a similar workaround into RCU's quiescent-state
> forcing logic.
>
> But how should this really be done?
>
> Isn't there some sort of monitoring of nohz_full CPUs for accounting
> purposes?  If so, would it make sense for this monitoring to check for
> long-duration kernel execution and enable the tick in this case?  The
> RCU dyntick machinery can be used to remotely detect the long-duration
> kernel execution using something like the following:
>
>         int nohz_in_kernel_snap = rcu_dynticks_snap_cpu(cpu);
>
>         ...
>
>         if (rcu_dynticks_in_eqs_cpu(cpu, nohz_in_kernel_snap)
>                 nohz_in_kernel_snap = rcu_dynticks_snap_cpu(cpu);
>         else
>                 /* Turn on the tick! */

This solution does make sense to me and is simpler than the
rcu_nmi_exit_common() solution you proposed in the other thread.

Though I am not sure about the intricacies of remotely enabling the
timer tick for a CPU. But overall, the above solution does seem
simpler.

thanks,

 - Joel