linux-kernel - Re: [RFC PATCH v2 0/2] sched/fair: Choose the CPU where short task is running during wake up

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <349903d8-3951-7d85-82e4-2e32e83ba0a3@amd.com>
Date:   Fri, 2 Dec 2022 08:51:23 +0530
From:   K Prateek Nayak <kprateek.nayak@....com>
To:     Chen Yu <yu.c.chen@...el.com>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Tim Chen <tim.c.chen@...el.com>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Juri Lelli <juri.lelli@...hat.com>,
        Rik van Riel <riel@...riel.com>,
        Aaron Lu <aaron.lu@...el.com>,
        Abel Wu <wuyun.abel@...edance.com>,
        Yicong Yang <yangyicong@...ilicon.com>,
        "Gautham R . Shenoy" <gautham.shenoy@....com>,
        Ingo Molnar <mingo@...hat.com>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Valentin Schneider <vschneid@...hat.com>,
        Hillf Danton <hdanton@...a.com>,
        Honglei Wang <wanghonglei@...ichuxing.com>,
        Len Brown <len.brown@...el.com>,
        Chen Yu <yu.chen.surf@...il.com>, linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH v2 0/2] sched/fair: Choose the CPU where short task is
 running during wake up

Hello Chenyu,

On 11/30/2022 9:33 AM, Chen Yu wrote:
> On 2022-11-22 at 16:01:42 +0530, K Prateek Nayak wrote:
>> Hello Chenyu,
>>
>> I've tested v2 series on an dual socket Zen3 system (2 x 64C/128T) and
>> the results are largely positive.
>>
> Thank you Prateek, and sorry for late response. 

Thank you for taking a look at the report.

>> tl;dr
>>
>> o Hackbench results are mostly similar with tip.
>> o schbench sees improvements to tail latency when the system is
>>   loaded in NPS1 case but I do see one small regression for
>>   128 workers in NPS4 mode.
>> o tbench sees small gains in NPS2 and NPS4 mode
>> o Stream and Spec-JBB results remain same as the tip.
>> o ycsb-mongodb sees small gains in NPS2 and NPS4 mode.
>> o unixbench results see small to moderate gains overall.
>>
>> I'll leave the detailed results below:
>>
>> [..snip..]
>>
>> Note: On the tested kernel, with 128 clients, tbench can
>> run into a bottleneck during C2 exit. More details can be
>> found at:
>> https://lore.kernel.org/lkml/20220921063638.2489-1-kprateek.nayak@amd.com/
>> This issue has been fixed in v6.0 but was not part of the
>> tip kernel when I started testing. This data point has
>> been rerun with C2 disabled to get representative results.
> This reminds me that, previously I tested with Cstates > C1 disabled, and
> with turbo disabled, so as to mitigate possible deviation. May I know if
> all C-states and turbo are enabled in your test besides tbench?

I do run with all C-states and turbo enabled with performance governor.
I can do a parallel run with C2 and turbo disabled to bring down any
possibility of external factors affecting the results. In the past,
we've seen some issues come to light when running with C2 and turbo
enabled so I had stuck to it. Thank you for pointing this out.

>>
>> [..snip..]
>>
>> Except for schbench with 128 workers in NPS4 mode, I do not
>> see any large regressions for the above workloads and I do
>> see small to moderate gains overall for most workload, even
>> the larger ones. I'll try to get data for more workload but
>> overall the idea seems promising. I'll also get some numbers
>> with the changes Peter suggested on Patch 1.
> I spent sometime to dig into the issue which motivates me to propose this
> solution. And it was found that this issue could not be easily solved
> directly because there seems to be an inevitable race condition window, with the
> increasing of CPU number, this race condition is exposed easlier. So
> current patch is an indirect solution to avoid that, I'll send the detail
> in v3. 

I see that v3 is out. Thank you for the detailed explanation and the
visualization of the bottleneck in v3 Patch 2.

>>
>> If there is any specific workload you would like me to run
>> on the test machine, please do let me know.
> Thanks for always helping us to test the patch, I'll send v3 once I get the
> result and we can discuss on that then.

I've queued up runs for v3 with the same set of benchmarks reported
above. I will make a point to include results with C2 and turbo disabled
to reduce external variables.
I'll share the results on v3 in the coming week.
--
Thanks and Regards,
Prateek