linux-kernel - Re: [PATCH 2/2] nohz: make nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 04 Apr 2015 05:43:09 +0200
From:	Mike Galbraith <umgwanakikbuti@...il.com>
To:	Chris Metcalf <cmetcalf@...hip.com>
Cc:	"Peter Zijlstra (Intel)" <peterz@...radead.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	"Rafael J. Wysocki" <rafael.j.wysocki@...el.com>,
	Martin Schwidefsky <schwidefsky@...ibm.com>,
	Ingo Molnar <mingo@...hat.com>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 2/2] nohz: make nohz_full imply isolcpus

On Sat, 2015-04-04 at 04:03 +0200, Mike Galbraith wrote:
> On Fri, 2015-04-03 at 15:21 -0400, Chris Metcalf wrote:
> > On 04/03/2015 02:08 PM, Mike Galbraith wrote:
> > > On Fri, 2015-04-03 at 12:24 -0400,  cmetcalf@...hip.comwrote:
> > > > From: Chris Metcalf <cmetcalf@...hip.com>
> > > > 
> > > > It's not clear that nohz_full is useful without isolcpus also
> > > > set, since otherwise the scheduler has to run periodically to
> > > > try to determine whether to steal work from other cores.
> > > > 
> > > > Signed-off-by: Chris Metcalf <cmetcalf@...hip.com>
> > > Ack!  nohz_full= as currently defined makes zero sense when the 
> > > cpu
> > > set (which should be spelled cpuset) remains connected to the
> > > scheduler.  Perturbation of tasks to PREVENT cpu domination is 
> > > what
> > > the scheduler does for a living.  Sprinkling microsecond savers 
> > > all
> > > over the kernel is pretty silly if you don't shut down the 
> > > mother 
> > > lode
> > > of perturbation.
> > 
> > Sounds like a thumbs up for this patch, then?  :-)
> 
> Yup.  The other thumb turns in the up direction when folks start 
> spelling cpuset properly ;-)  Static isolcpus was supposed to go 
> away.

Speaking of microsecond savers, the (ick) deferment experiment below 
cut 60 core jitter in half.  Shooting the clocksource watchdog fixes 
alternating ~15us/~5us tick on my desktop box.

With workqueue twiddles and whatnot floating around, the thing is 
starting to look viable.

---
 kernel/sched/core.c       |    5 +++--
 kernel/time/clocksource.c |    5 +++++
 2 files changed, 8 insertions(+), 2 deletions(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2604,12 +2604,13 @@ u64 scheduler_tick_max_deferment(void)
        struct rq *rq = this_rq();
        unsigned long next, now = ACCESS_ONCE(jiffies);
 
-       next = rq->last_sched_tick + HZ;
+       next = (rq->last_sched_tick + HZ) | (rq->clock & 0x3f);
 
        if (time_before_eq(next, now))
                return 0;
 
-       return jiffies_to_nsecs(next - now);
+       /* Add noise to avoid CPUs colliding at tick boundaries */
+       return jiffies_to_nsecs(next - now) | (rq->clock & 0xfffff);
 }
 #endif
 
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -267,8 +267,13 @@ static void clocksource_watchdog(unsigne
         * to each other.
         */
        next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask);
+skip_nohz_full:
        if (next_cpu >= nr_cpu_ids)
                next_cpu = cpumask_first(cpu_online_mask);
+       if (next_cpu && tick_nohz_full_cpu(next_cpu)) {
+               next_cpu = cpumask_next(next_cpu, cpu_online_mask);
+               goto skip_nohz_full;
+       }
        watchdog_timer.expires += WATCHDOG_INTERVAL;
        add_timer_on(&watchdog_timer, next_cpu);
 out:
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/