linux-kernel - Re: [ANNOUNCE] 3.7-nohz1

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAFTL4hy_k8rWg5EAFnh-owey7NJCQbK+ULrkNrSjgdynzEseTw@mail.gmail.com>
Date:	Mon, 24 Dec 2012 00:43:25 +0100
From:	Frederic Weisbecker <fweisbec@...il.com>
To:	Steven Rostedt <rostedt@...dmis.org>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Alessio Igor Bogani <abogani@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Avi Kivity <avi@...hat.com>,
	Chris Metcalf <cmetcalf@...era.com>,
	Christoph Lameter <cl@...ux.com>,
	Geoff Levand <geoff@...radead.org>,
	Gilad Ben Yossef <gilad@...yossef.com>,
	Hakan Akkan <hakanakkan@...il.com>,
	Ingo Molnar <mingo@...nel.org>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Paul Gortmaker <paul.gortmaker@...driver.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Li Zhong <zhong@...ux.vnet.ibm.com>
Subject: Re: [ANNOUNCE] 3.7-nohz1

2012/12/21 Steven Rostedt <rostedt@...dmis.org>:
> On Thu, 2012-12-20 at 19:32 +0100, Frederic Weisbecker wrote:
>> Let's imagine you have 4 CPUs. We keep the CPU 0 to offline RCU callbacks there and to
>> handle the timekeeping. We set the rest as full dynticks. So you need the following kernel
>> parameters:
>>
>>       rcu_nocbs=1-3 full_nohz=1-3
>>
>> (Note rcu_nocbs value must always be the same as full_nohz).
>
> Why? You can't have: rcu_nocbs=1-4 full_nohz=1-3

That should be allowed.

>   or: rcu_nocbs=1-3 full_nohz=1-4 ?

But that not.

You need to have: rcu_nocbs & full_nohz == full_nohz. This is because
the tick is not there to maintain the local RCU callbacks anymore. So
this must be offloaded to the rcu_nocb threads.

I just have a doubt with rcu_nocb. Do we still need the tick to
complete the grace period for local rcu callbacks? I need to discuss
that with Paul.

>
> That needs to be fixed. Either with a warning, and/or to force the two
> to be the same. That is, if they specify:
>
>   rcu_nocbs=1-3 full_nohz=1-4
>
> Then set rcu_nocbs=1-4 with a warning about it. Or simply set
>  full_nohz=1-3.

Yep, will do.

Thanks!

>
> -- Steve
>
>>
>> Now if you want proper isolation you need to:
>>
>> * Migrate your processes adequately
>> * Migrate your irqs to CPU 0
>> * Migrate the RCU nocb threads to CPU 0. Example with the above configuration:
>>
>>       for p in $(ps -o pid= -C rcuo1,rcuo2,rcuo3)
>>       do
>>               taskset -cp 0 $p
>>       done
>>
>> Then run what you want on the full dynticks CPUs. For best results, run 1 task
>> per CPU, mostly in userspace and mostly CPU bound (otherwise more IO = more kernel
>> mode execution = more chances to get IPIs, tick restarted, workqueues, kthreads, etc...)
>>
>> This page contains a good reminder for those interested in CPU isolation: https://github.com/gby/linux/wiki
>>
>> But keep in mind that my tree is not yet ready for serious production.
>>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/