linux-kernel - Re: [PATCH] nohz1: Documentation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130321185821.GF3637@linux.vnet.ibm.com>
Date:	Thu, 21 Mar 2013 11:58:21 -0700
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Christoph Lameter <cl@...ux.com>
Cc:	Steven Rostedt <rostedt@...dmis.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Rob Landley <rob@...dley.net>, linux-kernel@...r.kernel.org,
	josh@...htriplett.org, zhong@...ux.vnet.ibm.com,
	khilman@...aro.org, geoff@...radead.org, tglx@...utronix.de
Subject: Re: [PATCH] nohz1: Documentation

On Thu, Mar 21, 2013 at 06:39:09PM +0000, Christoph Lameter wrote:
> On Thu, 21 Mar 2013, Paul E. McKenney wrote:
> 
> > > Why are there multiple rcuo threads? Would a single thread that may be
> > > able to run on multiple cpus not be sufficient?
> >
> > In many cases, this would indeed be sufficient.  However, if you have
> > enough CPUs posting RCU callbacks, then the single thread would become
> > a bottleneck, eventually resulting in an OOM.  Per-CPU kthreads avoid
> > this possibility.
> 
> Spawn another if the load gets too high for a single cpu?
> 
> > That said, if you know that your workload's RCU callbacks could be
> > serviced by a single CPU, you can bind all the rcuo kthreads to a
> > single CPU.
> 
> Yeah doing that right now but I'd like to see it handled without manual
> intervention.

Given that RCU has no idea where you want them to run, some manual
intervention would most likely be required even if RCU spawned them
dynamically, right?

> > > > Again, good point!
> > >
> > > Uggh. That will cause problems and did cause problems when I tried to use
> > > nohz.
> > >
> > > The OS always has some sched other tasks around that become runnable after
> > > a while (like for example the vm statistics update, or the notorious slab
> > > scanning). As long as SCHED_FIFO is active and there is no process in the
> > > same scheduling class then tick needs to be off. Also wish that this would
> > > work with SCHED_OTHER if there is only a single task with a certain renice
> > > value (-10?) and the rest is runnable at lower priorities. Maybe in that
> > > case stop the tick for a longer period and then give the lower priority
> > > tasks a chance to run but then switch off the tick again.
> >
> > These sound to me like good future enhancements.
> >
> > In the meantime, one approach is to bind all these SCHED_OTHER tasks
> > to designated housekeeping CPU(s) that don't run your main workload.
> 
> One cannot bind kevent threads and other per cpu threads to another
> processor. So right now there is no way to avoid this issue.

Yep, my approach works only for those threads that are free to migrate.
Of course, in some cases, you can avoid per-CPU threads running by pinning
interrupts, avoiding certain operations in your workload, and so on.

So, again, removing scheduling-clock interrupts in more situations is
a good future enhancement.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/