linux-kernel - Re: [Resend PATCH] psi : calc cfs task memstall time more precisely

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <05a2e61e-9c55-8f8d-5e72-9854613e53c9@arm.com>
Date:   Tue, 9 Nov 2021 10:43:37 +0100
From:   Dietmar Eggemann <dietmar.eggemann@....com>
To:     Xuewen Yan <xuewen.yan94@...il.com>
Cc:     Zhaoyang Huang <huangzhaoyang@...il.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Michal Hocko <mhocko@...nel.org>,
        Vladimir Davydov <vdavydov.dev@...il.com>,
        Zhaoyang Huang <zhaoyang.huang@...soc.com>,
        "open list:MEMORY MANAGEMENT" <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        xuewen.yan@...soc.com, Ke Wang <Ke.Wang@...soc.com>
Subject: Re: [Resend PATCH] psi : calc cfs task memstall time more precisely

On 08/11/2021 09:49, Xuewen Yan wrote:
> Hi Dietmar
> 
> On Sat, Nov 6, 2021 at 1:20 AM Dietmar Eggemann
> <dietmar.eggemann@....com> wrote:
>>
>> On 05/11/2021 06:58, Zhaoyang Huang wrote:
>>>> I don't understand the EAS (probably asymmetric CPU capacity is meant
>>>> here) angle of the story. Pressure on CPU capacity which is usable for
>>>> CFS happens on SMP as well?
>>>  Mentioning EAS here mainly about RT tasks preempting small CFS tasks
>>> (big CFS tasks could be scheduled to big core), which would introduce
>>> more proportion of preempted time within PSI_MEM_STALL than SMP does.
>>
>> What's your CPU layout? Do you have the little before the big CPUs? Like
>> Hikey 960?

[...]

>> And I guess rt class prefers lower CPU numbers hence you see this?
>>
> our CPU layout is:
> xuewen.yan:/ # cat /sys/devices/system/cpu/cpu*/cpu_capacity
> 544
> 544
> 544
> 544
> 544
> 544
> 1024
> 1024
> 
> And in our platform, we use the kernel in mobile phones with Android.
> And we prefer power, so we prefer the RT class to run on little cores.

Ah, OK, out-of-tree extensions.

[...]

>>>>>>>> +     if (current->in_memstall)
>>>>>>>> +             growth_fixed = div64_ul((1024 - rq->avg_rt.util_avg - rq->avg_dl.util_avg
>>>>>>>> +                                     - rq->avg_irq.util_avg + 1) * growth, 1024);
>>>>>>>> +
>>>>
>>>> We do this slightly different in scale_rt_capacity() [fair.c]:
>>>>
>>>> max = arch_scale_cpu_capacity(cpu_of(rq) /* instead of 1024 to support
>>>>                                             asymmetric CPU capacity */
>>> Is it possible that the SUM of rqs' util_avg large than
>>> arch_scale_cpu_capacity because of task migration things?
>>
>> I assume you meant if the rq (cpu_rq(CPUx)) util_avg sum (CFS, RT, DL,
>> IRQ and thermal part) can be larger than arch_scale_cpu_capacity(CPUx)?
>>
>> Yes it can.
>>
>> Have a lock at
>>
>> effective_cpu_util(..., max, ...) {
>>
>>   if (foo >= max)
>>     return max;
>>
>> }
>>
>> Even the CFS part (cpu_rq(CPUx)->cfs.avg.util_avg) can be larger than
>> the original cpu capacity (rq->cpu_capacity_orig).
>>
>> Have a look at cpu_util(). capacity_orig_of(CPUx) and
>> arch_scale_cpu_capacity(CPUx) both returning rq->cpu_capacity_orig.
>>
> 
> Well, your means is we should not use the 1024 and should use the
> original cpu capacity?
> And maybe use the "sched_cpu_util()" is a good choice just like this:
> 
> +       if (current->in_memstall)
> +               growth_fixed = div64_ul(cpu_util_cfs(rq) * growth,
> sched_cpu_util(rq->cpu, capacity_orig_of(rq->cpu)));

Not sure about this. In case util_cfs=0 you would get scale=0.

IMHO, you need

cap      = rq->cpu_capacity
cap_orig = rq->cpu_capacity_orig

scale = (cap * X) / cap_orig

or if the update of these rq values happens to infrequently for you then
you have to calc the pressure evey time. Something like:

pressure = cpu_util_rt(rq) + cpu_util_dl(rq)

irq = cpu_util_irq(rq)

if (irq >= cap_orig)
    pressure = cap_orig
else
    pressure = scale_irq_capacity(pressure, irq, cap_orig)
    pressure += irq

scale = ((cap_orig - pressure) * X) / cap_orig