linux-kernel - Re: [PATCH] sched/fair: Skip newidle_balance() when an idle CPU is woken up to process an IPI

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Tue, 23 Jan 2024 10:28:31 +0530
From: K Prateek Nayak <kprateek.nayak@....com>
To: Tim Chen <tim.c.chen@...ux.intel.com>, linux-kernel@...r.kernel.org
Cc: mingo@...hat.com, peterz@...radead.org, juri.lelli@...hat.com,
 vincent.guittot@...aro.org, dietmar.eggemann@....com, rostedt@...dmis.org,
 bsegall@...gle.com, mgorman@...e.de, bristot@...hat.com,
 vschneid@...hat.com, gautham.shenoy@....com,
 David Vernet <void@...ifault.com>
Subject: Re: [PATCH] sched/fair: Skip newidle_balance() when an idle CPU is
 woken up to process an IPI

Hello Tim,

On 1/23/2024 3:29 AM, Tim Chen wrote:
> On Fri, 2024-01-19 at 14:15 +0530, K Prateek Nayak wrote:
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index b803030c3a03..1fedc7e29c98 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -8499,6 +8499,16 @@ done: __maybe_unused;
>>  	if (!rf)
>>  		return NULL;
>>  
>> +	/*
>> +	 * An idle CPU in TIF_POLLING mode might end up here after processing
>> +	 * an IPI when the sender sets the TIF_NEED_RESCHED bit and avoids
>> +	 * sending an actual IPI. In such cases, where an idle CPU was woken
>> +	 * up only to process an interrupt, without necessarily queuing a task
>> +	 * on it, skip newidle_balance() to facilitate faster idle re-entry.
>> +	 */
>> +	if (prev == rq->idle)
>> +		return NULL;
>> +
> 
> Should we check the call function queue directly to detect that there is
> an IPI waiting to be processed? something like
> 
> 	if (!llist_empty(&per_cpu(call_single_queue, rq->cpu)))
> 		return NULL;

That could be a valid check too. However, if an IPI is queued right
after this check, the processing is still delayed since
newidle_balance() only bails out for scenarios when a wakeup is trying
to queue a new task on the CPU running the newidle_balance().

> 
> Could there be cases where we want to do idle balance in this code path?
> Say a cpu is idle and a scheduling tick came in, we may try
> to look for something to run on the idle cpu.  Seems like after
> your change above, that would be skipped.

Wouldn't scheduler_tick() do load balancing when the time comes? In my
testing, I did not see a case where the workloads I tested were
sensitive to the aspect of newidle_balance() being invoked at scheduler
tick. Have you come across a workload which might be sensitive to this
aspect that I can quickly test and verify? Meanwhile, I'll run the
workloads mentioned in the commit log on an Intel system to see if I
can spot any sensitivity to this change.

Adding David to the thread too since HHVM seems to be one of those
workloads that is very sensitive to a successful newidle_balance().

> 
> Tim
> 
> 
>>  	new_tasks = newidle_balance(rq, rf);
>>  
>>  	/*
> 

--
Thanks and Regards,
Prateek