lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <b0dea973-bf41-4f44-850c-7bc860d5bf8a@nvidia.com>
Date: Mon, 17 Mar 2025 23:07:36 +0100
From: Joel Fernandes <joelagnelf@...dia.com>
To: Tejun Heo <tj@...nel.org>
Cc: linux-kernel@...r.kernel.org, David Vernet <void@...ifault.com>,
 Andrea Righi <arighi@...dia.com>, Changwoo Min <changwoo@...lia.com>,
 Ingo Molnar <mingo@...hat.com>, Peter Zijlstra <peterz@...radead.org>,
 Juri Lelli <juri.lelli@...hat.com>,
 Vincent Guittot <vincent.guittot@...aro.org>,
 Dietmar Eggemann <dietmar.eggemann@....com>,
 Steven Rostedt <rostedt@...dmis.org>, Ben Segall <bsegall@...gle.com>,
 Mel Gorman <mgorman@...e.de>, Valentin Schneider <vschneid@...hat.com>
Subject: Re: [PATCH RFC] sched_ext: Choose prev_cpu if idle and cache affine
 without WF_SYNC



On 3/17/2025 6:08 PM, Tejun Heo wrote:
> Hello, Joel.
> 
> On Mon, Mar 17, 2025 at 04:28:02AM -0400, Joel Fernandes wrote:
>> Consider that the previous CPU is cache affined to the waker's CPU and
>> is idle. Currently, scx's default select function only selects the
>> previous CPU in this case if WF_SYNC request is also made to wakeup on the
>> waker's CPU.
>>
>> This means, without WF_SYNC, the previous CPU being cache affined to the
>> waker and is idle is not considered. This seems extreme. WF_SYNC is not
>> normally passed to the wakeup path outside of some IPC drivers but it is
>> very possible that the task is cache hot on previous CPU and shares
>> cache with the waker CPU. Lets avoid too many migrations and select the
>> previous CPU in such cases.
> 
> Hmm.. if !WF_SYNC:
> 
> 1. If smt, if prev_cpu's core is idle, pick it. If not, try to pick an idle
>    core in widening scopes.
> 
> 2. If no idle core is foudn, pick prev_cpu if idle. If not, search for an
>    idle CPU in widening scopes.
> 
> So, it is considering prev_cpu, right?

Yes, but unlike the FAIR scheduler, it is not selecting the previous idle SMT
for a busy core, and instead looking for a fully idle core first. I saw that as
a difference between what FAIR and EXT do.


> I think it's preferring idle core a
> bit too much - it probably doesn't make sense to cross the NUMA boundary if
> there is an idle CPU in this node, at least.
> 

Yes, that is a bit extreme. I believe that is what my patch is fixing.  If
previous CPU and current CPU share cache, we prefer busy cores with free
previous idle SMT, otherwise go looking for fully idle cores. But it sounds like
from Peter's reply that is not necessarily a good thing to do in case 'fast
numa'. So I guess there is no good answer (?) or way of doing it.

I guess one of things I would like is for FAIR and EXT's selection algorithms
(default in EXT's case) are close in behavior and consistent. And maybe we can
unify their behaviors in this regard, and 'do the right thing' for majority of
workloads and machines.

> Isn't the cpus_share_cache() code block mostly about not doing
> waker-affining if prev_cpu of the wakee is close enough and idle, so
> waker-affining is likely to be worse?

It seems with FAIR, we do waker-affining first:

In select_idle_sibling(...):
	if ((available_idle_cpu(target) || sched_idle_cpu(target)) &&
	    asym_fits_cpu(task_util, util_min, util_max, target))
		return target;


before checking if previous CPU shares cache:

In select_idle_sibling(...):
	/*
	 * If the previous CPU is cache affine and idle, don't be stupid:
	 */
	if (prev != target && cpus_share_cache(prev, target) &&
	    (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
	    asym_fits_cpu(task_util, util_min, util_max, prev)) {

		if (!static_branch_unlikely(&sched_cluster_active) ||
		    cpus_share_resources(prev, target))
			return prev;

		prev_aff = prev;
	}

So I think that block is not about "not doing waker affining" but about "doing
wakee affining".

thanks,

 - Joel




Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ