linux-kernel - Re: [External] Re: [PATCH] sched/fair: fix broken bandwidth control with nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YkHRmv/OcABIB0wP@hirez.programming.kicks-ass.net>
Date:   Mon, 28 Mar 2022 17:17:46 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Chengming Zhou <zhouchengming@...edance.com>
Cc:     mingo@...hat.com, juri.lelli@...hat.com,
        vincent.guittot@...aro.org, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de,
        bristot@...hat.com, linux-kernel@...r.kernel.org,
        duanxiongchun@...edance.com, songmuchun@...edance.com,
        Frederic Weisbecker <fweisbec@...il.com>
Subject: Re: [External] Re: [PATCH] sched/fair: fix broken bandwidth control
 with nohz_full

On Mon, Mar 28, 2022 at 09:50:05PM +0800, Chengming Zhou wrote:
> On 2022/3/28 21:20, Peter Zijlstra wrote:
> > On Mon, Mar 28, 2022 at 07:07:51PM +0800, Chengming Zhou wrote:
> >> With nohz_full enabled on cpu, the scheduler_tick() will be stopped
> >> when only one CFS task left on rq.
> >>
> >> scheduler_tick()
> >>   task_tick_fair()
> >>     entity_tick()
> >>       update_curr()
> >>         account_cfs_rq_runtime(cfs_rq, delta_exec) --> stopped
> >>
> >> So that running task can't account its runtime periodically, but
> >> the cfs_bandwidth hrtimer still __refill_cfs_bandwidth_runtime()
> >> periodically. Later in one period, the task would account very
> >> big delta_exec, which cause the cfs_rq to be throttled for a
> >> long time.
> >>
> >> There are two solutions for the problem, the first is that we
> >> can check in sched_can_stop_tick() if current task's cfs_rq
> >> have runtime_enabled, in which case we don't stop tick. But
> >> it will make nohz_full almost useless in cloud environment
> >> that every container has the cpu bandwidth control setting.
> > 
> > How is NOHZ_FULL useful in that environment to begin with? If you set
> > bandwidth crap, the expectation is that there is overcommit, which more
> > or less assumes lots of scheduling, presumably VMs or somesuch crud.
> > 
> > So how does NOHZ_FULL make sense?
> 
> Yes, we have scheduled some VMs in cgroups on the host, which
> enabled NOHZ_FULL to reduce the interference of tick to vcpu task
> if it's the only task running on cpu.
> 
> This problem will however throttle it wrongly, even if it hasn't
> used up its quota.
> 
> Do you suggest that we shouldn't stop tick when the current task's
> cfs_rq has runtime_enabled ?

I'm not suggesting anything just yet as I'm not sure I understand things
well enough. I'm just wondering if NOHZ_FULL makes sense for you since
NOHZ_FULL makes system entry/exit so much more expensive.

NOHZ_FULL is for use-cases that 'never' intend to go into the kernel,
your use-case actively relies on going into the kernel. Hence the
confusion.