linux-kernel - Re: [PATCH 4/5] sched/isolation: Residual 1Hz scheduler tick offload

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20171219091911.tg2k4w7mgv2bcmeb@hirez.programming.kicks-ass.net>
Date:   Tue, 19 Dec 2017 10:19:11 +0100
From:   Peter Zijlstra <peterz@...radead.org>
To:     Frederic Weisbecker <frederic@...nel.org>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        Chris Metcalf <cmetcalf@...lanox.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Luiz Capitulino <lcapitulino@...hat.com>,
        Christoph Lameter <cl@...ux.com>,
        "Paul E . McKenney" <paulmck@...ux.vnet.ibm.com>,
        Ingo Molnar <mingo@...nel.org>,
        Wanpeng Li <kernellwp@...il.com>,
        Mike Galbraith <efault@....de>, Rik van Riel <riel@...hat.com>
Subject: Re: [PATCH 4/5] sched/isolation: Residual 1Hz scheduler tick offload

On Tue, Dec 19, 2017 at 04:23:57AM +0100, Frederic Weisbecker wrote:
> When a CPU runs in full dynticks mode, a 1Hz tick remains in order to
> keep the scheduler stats alive. However this residual tick is a burden
> for Real-Time tasks that can't stand no interruption at all.

I'm not sure that is accurate. RT doesn't necessarily have anything much
to so with this. The tick is per definition very deterministic and thus
should not be a problem.

> Adding the boot parameter "isolcpus=nohz_offload" will now outsource
> these scheduler ticks to the global workqueue so that a housekeeping CPU
> handles that tick remotely.

The global workqueue sounds horrific; surely you want at least one such
housekeeping CPU per node or something ?

> Note it's still up to the user to affine the global workqueues to the
> housekeeping CPUs through /sys/devices/virtual/workqueue/cpumask or
> domains isolation.

Not sure I understand what this means... from what I can tell you're
using an unbound workqueue, there's no way to split the ticks up to node
local CPUs.

> +static void sched_tick_remote(struct work_struct *work)
> +{
> +	struct delayed_work *dwork = to_delayed_work(work);
> +	struct tick_work *twork = container_of(dwork, struct tick_work, work);
> +	struct rq *rq = cpu_rq(twork->cpu);
> +	struct rq_flags rf;
> +
> +	rq_lock_irq(rq, &rf);
> +	update_rq_clock(rq);
> +	rq->curr->sched_class->task_tick(rq, rq->curr, 0);
> +	rq_unlock_irq(rq, &rf);
> +
> +	queue_delayed_work(system_unbound_wq, dwork, HZ);
> +}
> +
> +void sched_tick_start(int cpu)
> +{
> +	struct tick_work *twork;
> +
> +	if (housekeeping_cpu(cpu, HK_FLAG_TICK_SCHED))
> +		return;
> +
> +	WARN_ON_ONCE(!tick_work_cpu);
> +
> +	twork = per_cpu_ptr(tick_work_cpu, cpu);
> +	twork->cpu = cpu;
> +	INIT_DELAYED_WORK(&twork->work, sched_tick_remote);
> +	queue_delayed_work(system_unbound_wq, &twork->work, HZ);
> +
> +	return;
> +}
> +
> +#ifdef CONFIG_HOTPLUG_CPU
> +void sched_tick_stop(int cpu)
> +{
> +	struct tick_work *twork;
> +
> +	if (housekeeping_cpu(cpu, HK_FLAG_TICK_SCHED))
> +		return;
> +
> +	WARN_ON_ONCE(!tick_work_cpu);
> +
> +	twork = per_cpu_ptr(tick_work_cpu, cpu);
> +	cancel_delayed_work_sync(&twork->work);
> +
> +	return;
> +}
> +#endif /* CONFIG_HOTPLUG_CPU */

This seems daft in that you _always_ run this remote tick, even when the
CPU in question is not in nohz (full) mode.