linux-kernel - Re: [External] Re: [PATCH] sched/fair: fix broken bandwidth control with nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <933ecd2d-fe2f-3b52-f2f0-c5b1a132dd81@bytedance.com>
Date:   Tue, 29 Mar 2022 00:35:11 +0800
From:   Chengming Zhou <zhouchengming@...edance.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     mingo@...hat.com, juri.lelli@...hat.com,
        vincent.guittot@...aro.org, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de,
        bristot@...hat.com, linux-kernel@...r.kernel.org,
        duanxiongchun@...edance.com, songmuchun@...edance.com,
        Frederic Weisbecker <fweisbec@...il.com>
Subject: Re: [External] Re: [PATCH] sched/fair: fix broken bandwidth control
 with nohz_full

On 2022/3/28 23:56, Peter Zijlstra wrote:
> On Mon, Mar 28, 2022 at 11:40:25PM +0800, Chengming Zhou wrote:
> 
>>> NOHZ_FULL is for use-cases that 'never' intend to go into the kernel,
>>> your use-case actively relies on going into the kernel. Hence the
>>> confusion.
>>
>> In fact, I put a testcase at the end of git message, in which only run
>> a userspace loop workload:
>>
>> cd /sys/fs/cgroup
>> echo "+cpu" > cgroup.subtree_control
>>
>> mkdir test
>> echo "105000 100000" > test/cpu.max
>>
>> echo $$ > test/cgroup.procs
>> taskset -c 1 bash -c "while true; do let i++; done"  --> will be throttled
> 
> Ofcourse.. I'm arguing that bandiwdth control and NOHZ_FULL are somewhat
> mutually exclusive, use-case wise. So I really don't get why you'd want
> them both.

This problem is found by our VM team, they use bandwidth for overcommit,
share CPUs between two VMs.

> 
> NOHZ_FULL says, "I 'never' intend to go to the kernel"

Like VCPU seldom kvm_exit to the kernel, stop tick is helpful for performance,
since kvm_exit is more expensive.

> 
> bandwidth control says: "I expect to be sharing the system and must be
> interrupted to not consume too much time", which very much implies: "I
> will go into the kernel".

Yes, agree. If the tasks in the task_group used up quota, they have to
go into the kernel to resched out.

> 
> The trade-off we make to make NOHZ_FULL work, makes system enter/exit
> *far* more expensive. There's also people asking to outright kill a task
> that causes entry under NOHZ_FULL.

It's correct that the task under NOHZ_FULL shouldn't often enter/exit.

> 
> So yes, you can configure it, but why does it make sense?

I don't know if other people have the same use-case, or is there other
better way to do VMs overcommit and bandwidth...

Thanks.