linux-kernel - Re: [External] Re: [PATCH] sched/fair: fix broken bandwidth control with nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <c0465b27-328a-1288-fb50-d4c6321c8b4d@bytedance.com>
Date:   Mon, 28 Mar 2022 21:50:05 +0800
From:   Chengming Zhou <zhouchengming@...edance.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     mingo@...hat.com, juri.lelli@...hat.com,
        vincent.guittot@...aro.org, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de,
        bristot@...hat.com, linux-kernel@...r.kernel.org,
        duanxiongchun@...edance.com, songmuchun@...edance.com
Subject: Re: [External] Re: [PATCH] sched/fair: fix broken bandwidth control
 with nohz_full

On 2022/3/28 21:20, Peter Zijlstra wrote:
> On Mon, Mar 28, 2022 at 07:07:51PM +0800, Chengming Zhou wrote:
>> With nohz_full enabled on cpu, the scheduler_tick() will be stopped
>> when only one CFS task left on rq.
>>
>> scheduler_tick()
>>   task_tick_fair()
>>     entity_tick()
>>       update_curr()
>>         account_cfs_rq_runtime(cfs_rq, delta_exec) --> stopped
>>
>> So that running task can't account its runtime periodically, but
>> the cfs_bandwidth hrtimer still __refill_cfs_bandwidth_runtime()
>> periodically. Later in one period, the task would account very
>> big delta_exec, which cause the cfs_rq to be throttled for a
>> long time.
>>
>> There are two solutions for the problem, the first is that we
>> can check in sched_can_stop_tick() if current task's cfs_rq
>> have runtime_enabled, in which case we don't stop tick. But
>> it will make nohz_full almost useless in cloud environment
>> that every container has the cpu bandwidth control setting.
> 
> How is NOHZ_FULL useful in that environment to begin with? If you set
> bandwidth crap, the expectation is that there is overcommit, which more
> or less assumes lots of scheduling, presumably VMs or somesuch crud.
> 
> So how does NOHZ_FULL make sense?

Yes, we have scheduled some VMs in cgroups on the host, which
enabled NOHZ_FULL to reduce the interference of tick to vcpu task
if it's the only task running on cpu.

This problem will however throttle it wrongly, even if it hasn't
used up its quota.

Do you suggest that we shouldn't stop tick when the current task's
cfs_rq has runtime_enabled ?

Thanks.