Message-ID: <c66981ec-ebe9-3678-22a8-a0bdd3f8326b@didichuxing.com>
Date:   Sat, 11 Dec 2021 00:34:40 +0800
From:   Honglei Wang <wanghonglei@...ichuxing.com>
To:     Peter Zijlstra <peterz@...radead.org>
CC:     Ingo Molnar <mingo@...hat.com>, Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        "Daniel Bristot de Oliveira" <bristot@...hat.com>,
        <linux-kernel@...r.kernel.org>,
        Huaixin Chang <changhuaixin@...ux.alibaba.com>,
        Honglei Wang <jameshongleiwang@....com>
Subject: Re: [PATCH v2 2/3] sched/fair: prevent cpu burst too many periods



On 2021/12/9 21:08, Peter Zijlstra wrote:
> On Wed, Dec 08, 2021 at 10:50:38PM +0800, Honglei Wang wrote:
>> Tasks might get more cpu than quota in persistent periods due to the
>> cpu burst introduced by commit f4183717b370 ("sched/fair: Introduce the
>> burstable CFS controller").
> 
>> For example, one task group whose quota is
>> 100ms per period and can get 100ms burst, and its avg utilization is
>> around 105ms per period.
> 
> That would be a mis-configuration, surely..
> 

Well, it's a lame example to describe the spreading of the burst..

>> Once this group gets a free period which
>> leaves enough runtime, it has a chance to get computting power more
>> than its quota for 10 periods or more in common bandwidth configuration
>> (say, 100ms as period).
> 
> Sure, if it, for some miraculous reason, decides to sleep for a whole
> period and then resume, it can indeed consume up to that 100ms extra,
> which, if as per the above, done at 5ms per perios, would be 20 periods
> until depleted.
> 
>> It means tasks can 'steal' the bursted power to
>> do daily jobs because all tasks could be scheduled out or sleep to help
>> the group get free periods.
> 
> That's the design,,
> 
>> I believe the purpose of cpu burst is to help handling bursty worklod.
>> But if one task group can get computting power more than its quota for
>> persistent periods even there is no bursty workload, it's kinda broke.
> 
> So if that was were bursty, it could consume that 100ms extra in a
> single go and that would be fine, but spreading that same amount over 20
> periods is somehow a problem? -- even though the interference is less.
> 

The key thought I make the change is that If the spreading of burst 
power always happen, it indicates the quota is not comfortable, the 
better way is to do quota re-config for the container, but not always 
get extra power from spreading the burst part which is not in the 
consideration of high level container dispatcher such as k8s scheduler. 
Container dispatcher might dispatch jobs oversale. The containers get 
more power from the burst spreading is outside the sense of the 
dispatcher. It might mislead the estimation of the entire ability of the 
host.

IMO, cpu burst should be focus on the real burst workload, sharp and 
short term.

Well, if 2 periods are a bit short for some huge cpu calculation jobs, 
maybe we can add an option to define the burstable periods to let the 
user make the decision based on the workload if you think this limit 
periods idea make sense.

Thanks,
Honglei

>> This patch limits the burst to 2 periods so that it won't break the
>> quota limit for long. Permitting 2 periods can help on the scenario that
>> periods refresh lands in the middle of a burst workload. With this, we
>> can give task group more cpu burst power to handle the real burst
>> workload and don't worry about the 'stealing'.
> 
> I've yet so see an actual reason for any of this...
> 

Powered by blists - more mailing lists