linux-kernel - Re: [RFC PATCH 1/1] sched: Extend cpu idle state for 1ms

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 1 Aug 2023 15:42:02 +0800
From:   Aaron Lu <aaron.lu@...el.com>
To:     Chen Yu <yu.c.chen@...el.com>
CC:     Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        Shrikanth Hegde <sshegde@...ux.vnet.ibm.com>,
        <linux-kernel@...r.kernel.org>, Ingo Molnar <mingo@...hat.com>,
        Valentin Schneider <vschneid@...hat.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        "Vincent Guittot" <vincent.guittot@...aro.org>,
        Juri Lelli <juri.lelli@...hat.com>,
        Swapnil Sapkal <Swapnil.Sapkal@....com>, <x86@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Srikar Dronamraju <srikar@...ux.vnet.ibm.com>
Subject: Re: [RFC PATCH 1/1] sched: Extend cpu idle state for 1ms

On Thu, Jul 27, 2023 at 01:04:13PM +0800, Chen Yu wrote:
> On 2023-07-26 at 10:07:30 -0400, Mathieu Desnoyers wrote:
> > On 7/26/23 04:04, Shrikanth Hegde wrote:
> > > 
> > > 
> > > On 7/26/23 1:00 AM, Mathieu Desnoyers wrote:
> > > > Allow select_task_rq to consider a cpu as idle for 1ms after that cpu
> > > > has exited the idle loop.
> > > > 
> > > > This speeds up the following hackbench workload on a 192 cores AMD EPYC
> > > > 9654 96-Core Processor (over 2 sockets):
> > > > 
> > > > hackbench -g 32 -f 20 --threads --pipe -l 480000 -s 100
> > > > 
> > > > from 49s to 34s. (30% speedup)
> > > > 
> > > > My working hypothesis for why this helps is: queuing more than a single
> > > > task on the runqueue of a cpu which just exited idle rather than
> > > > spreading work over other idle cpus helps power efficiency on systems
> > > > with large number of cores.
> > > >
> 
> This looks interesting. And it does help power efficiency but how it could
> improve throughput? Is it because of hot cache locality waking up task on
> it previous running CPU(because it will be easier to be treated as idle),
> or just reducing the time in select_idle_sibling()?
>  

According to my tests on Intel SPR, part of the reason is reduced
migration cost due to tg->load_avg. I think it has similar effect as
your previous patch(wake up short task on current CPU):
https://lore.kernel.org/lkml/cover.1682661027.git.yu.c.chen@intel.com/
Both can reduce task migration somehow.

Thanks,
Aaron

> > Good point !
> > 
> > Can you try your benchmark replacing the if () statement above by:
> > 
> > +       if (sched_clock() < READ_ONCE(rq->idle_end_time) + IDLE_CPU_DELAY_NS &&
> > +           READ_ONCE(rq->nr_running) <= 4)
> 
> If I understand correctly, this nr_running is to filter the case that the system
> is saturated? If that is the case, maybe 
> 	rq->avg_idle >= sysctl_sched_migration_cost
> could be checked in case there is 1 long running task and we don't want to treat this
> cpu as 'idle'?
> 
> thanks,
> Chenyu