linux-kernel - Re: Inquiry: Should we remove "isolcpus= kernel boot option? (may have realtime uses)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 02 Jun 2008 11:39:34 -0700
From:	Max Krasnyansky <maxk@...lcomm.com>
To:	Dimitri Sivanich <sivanich@....com>
CC:	Paul Jackson <pj@....com>, linux-kernel@...r.kernel.org,
	Con Kolivas <kernel@...ivas.org>,
	"Derek L. Fults" <dfults@....com>, devik <devik@....cz>,
	Dinakar Guniguntala <dino@...ibm.com>,
	Emmanuel Pacaud <emmanuel.pacaud@...v-poitiers.fr>,
	Frederik Deweerdt <deweerdt@...e.fr>,
	Ingo Molnar <mingo@...e.hu>,
	Matthew Dobson <colpatch@...ibm.com>,
	Nick Piggin <nickpiggin@...oo.com.au>, rostedt@...dmis.org,
	Oleg Nesterov <oleg@...sign.ru>,
	"Paul E. McKenney" <paulmck@...ibm.com>,
	Paul Menage <menage@...gle.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	"Randy.Dunlap" <rddunlap@...l.org>, suresh.b.siddha@...el.com,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: Inquiry: Should we remove "isolcpus= kernel boot option? (may
 have realtime uses)

Dimitri Sivanich wrote:
> Paul,
> 
> On Sun, Jun 01, 2008 at 09:30:19PM -0500, Paul Jackson wrote:
>>     Do you, or someone you know, use "isolcpus="?
> 
> We use it.
> 
>>     Can we remove it?
> 
> We use isolcpus to ensure that boot-time intialization, specifically timer
> initialization, happens on a specific set of cpus that we won't be using for
> lower latency purposes. Some of these timers will repeatedly restart
> themselves on the same cpu and a few do add latency (although admittedly I
> haven't checked timer latency recently).
> 
> Looking at tracebacks in 2.6.26-rc3 from hrtimer_init() and
> internal_add_timer() things still appear to be working this way, with the
> timer starting on the originating cpu. If I isolate all but, say one, cpu,
> timers all seem to start on the unisolated cpu.
> 
> Attempts have been made to add an interface to ward timers off of specific
> cpus, but these have always been rejected.

Ah, I know exactly what you're talking about.  However this is non-issue these
days. In order to clear cpuN from all the timers and other things all you need
to do is to bring that cpu off-line
	echo 0 > /sys/devices/cpu/cpuN/online
and then bring it back online
	echo 1 > /sys/devices/cpu/cpuN/online

There are currently a couple of issues with scheduler domains and hotplug
event handling. I do have the fix for them, and Paul had already acked it.

btw Disabling scheduler load balancer is not enough.  Some timers are started
from the hard- and soft- irq handlers. Which means that you have to also
ensure that those CPUs do not handle any irqs (at least during
initialization). See my latest "default IRQ affinity" patch.

>>     Should we remove it?
> 
> Why?
Because the same functionality is available via more flexible mechanism that
is actively supported. isolcpus= is a static mechanism that requires reboots.
cpusets and cpu hotplug let you dynamically repartition the system at any time.
Also isolcpus= conflicts with the scheduler domains created by the cpusets.

>  
>>     Should we first deprecate it somehow, for a while, before
>>     removing it?
> 
> A better idea than just removing it.

I'd either nuke it or expose it when cpusets are disabled.
In other words
- if cpusets are enabled people should use cpusets to configure cpu resources.
- if cpusets are disabled then we could provide a sysctl (sched_balancer_mask
for example) that lets us control which cpus are balanced and which aren't.

Max

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/