linux-kernel - Re: [PATCH 6/6] sched/isolation: Tick offload documentation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20180214145248.GA2456@lerouge>
Date:   Wed, 14 Feb 2018 15:52:50 +0100
From:   Frederic Weisbecker <frederic@...nel.org>
To:     Ingo Molnar <mingo@...nel.org>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Chris Metcalf <cmetcalf@...lanox.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Luiz Capitulino <lcapitulino@...hat.com>,
        Christoph Lameter <cl@...ux.com>,
        "Paul E . McKenney" <paulmck@...ux.vnet.ibm.com>,
        Wanpeng Li <kernellwp@...il.com>,
        Mike Galbraith <efault@....de>, Rik van Riel <riel@...hat.com>
Subject: Re: [PATCH 6/6] sched/isolation: Tick offload documentation

On Fri, Feb 09, 2018 at 08:06:49AM +0100, Ingo Molnar wrote:
> 
> * Frederic Weisbecker <frederic@...nel.org> wrote:
> 
> > Update the documentation to reflect the 1Hz tick offload changes.
> > 
> > Signed-off-by: Frederic Weisbecker <frederic@...nel.org>
> > Cc: Chris Metcalf <cmetcalf@...lanox.com>
> > Cc: Christoph Lameter <cl@...ux.com>
> > Cc: Luiz Capitulino <lcapitulino@...hat.com>
> > Cc: Mike Galbraith <efault@....de>
> > Cc: Paul E. McKenney <paulmck@...ux.vnet.ibm.com>
> > Cc: Peter Zijlstra <peterz@...radead.org>
> > Cc: Rik van Riel <riel@...hat.com>
> > Cc: Thomas Gleixner <tglx@...utronix.de>
> > Cc: Wanpeng Li <kernellwp@...il.com>
> > Cc: Ingo Molnar <mingo@...nel.org>
> > ---
> >  Documentation/admin-guide/kernel-parameters.txt | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 39ac9d4..c851e41 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -1762,7 +1762,11 @@
> >  			specified in the flag list (default: domain):
> >  
> >  			nohz
> > -			  Disable the tick when a single task runs.
> > +			  Disable the tick when a single task runs. A residual 1Hz
> > +			  tick is offloaded to workqueues that you need to affine
> > +			  to housekeeping through the sysfs file
> > +			  /sys/devices/virtual/workqueue/cpumask or using the below
> > +			  domain flag.
> 
> This is pretty ambiguous and somewhat confusing, I'd suggest something like:
> 
> 			nohz
> 			  Disable the tick when a single task runs.
> 
> 			  A residual 1Hz tick is offloaded to workqueues, which you 
> 			  need to affine to housekeeping through the global 
> 			  workqueue's affinity configured via the 
> 			  /sys/devices/virtual/workqueue/cpumask sysfs file, or 
> 			  by using the 'domain' flag described below.
> 
> 			  NOTE: by default the global workqueue runs on all CPUs, 
> 			  so to protect individual CPUs the 'cpumask' file has to 
> 			  be configured manually after bootup.
> 
> Assuming what I wrote is correct - the CPU isolation config space is pretty 
> confusing all around and should be made a lot more human friendly ...

That's right. In fact "nohz_full=" affines the workqueues and it involves much
more: unbound timers affinity, RCU threads, etc...

So "nohz_full=" is the friendly interface as it does all in one.

Now the use of "isolcpus=" is supposed to be more finegrained and allow for more control.
Ideally I would like to have an "unbound" flag which affines all these unbound works. And
why not a "per_cpu" flag to disable or offload per cpu work such as watchdog.

So we would only need to do:

     isolcpus=nohz,unbound,per_cpu

Or even just:

     isolcpus=all

But before extending further isolcpus=, I would like to make sure I can make it mutable
later through cpusets. So this is work in progress.

Thanks.