linux-kernel - Re: [PATCH 1/6] sched/topology: introduce for_each_numa_hop_node() / sched_numa_hop

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z2R3eKPwBhzKU4y3@gpd3>
Date: Thu, 19 Dec 2024 20:43:52 +0100
From: Andrea Righi <arighi@...dia.com>
To: Yury Norov <yury.norov@...il.com>
Cc: Tejun Heo <tj@...nel.org>, David Vernet <void@...ifault.com>,
	Changwoo Min <changwoo@...lia.com>, Ingo Molnar <mingo@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Juri Lelli <juri.lelli@...hat.com>,
	Vincent Guittot <vincent.guittot@...aro.org>,
	Dietmar Eggemann <dietmar.eggemann@....com>,
	Steven Rostedt <rostedt@...dmis.org>,
	Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
	Valentin Schneider <vschneid@...hat.com>,
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH 1/6] sched/topology: introduce for_each_numa_hop_node() /
 sched_numa_hop_node()

On Thu, Dec 19, 2024 at 10:26:59AM -0800, Yury Norov wrote:
> On Wed, Dec 18, 2024 at 06:04:53AM -1000, Tejun Heo wrote:
> > Hello,
> > 
> > On Wed, Dec 18, 2024 at 11:23:40AM +0100, Andrea Righi wrote:
> > ...
> > > > So, this would work but given that there is nothing dynamic about this
> > > > ordering, would it make more sense to build the ordering and store it
> > > > per-node? Then, the iteration just becomes walking that array.
> > > 
> > > I've also considered doing that. I don't know if it'd work with offline
> > > nodes, but maybe we can just check node_online(node) at each iteration and
> > > skip those that are not online.
> 
> for_each_numa_hop_mask() only traverses N_CPU nodes, and N_CPU nodes have
> proper distances.
> 
> I think that for_each_numa_hop_node() should match for_each_numa_hop_mask().
> It would be good to cross-test them to ensure that they generate the same
> order at least for N_CPU nodes.

It'd be nice to have a kunit, I can take a look at this (in a separate
patch, I think we can add this later).

> 
> If you think that for_each_numa_hop_node() should traverse non-N_CPU nodes,
> you need a 'node_state' parameter. This will allow to make sure that at
> least N_CPU portion works correctly.
> 
> > Yeah, there can be e.g. for_each_possible_node_by_dist() wheke nodes with
> > unknown distances (offline ones?) are put at the end and then there's also
> > for_each_online_node_by_dist() which filters out offline ones, and the
> > ordering can be updated from a CPU hotplug callback.
> 
> We can assign UINT_MAX for those nodes I guess?
> 
> > The ordering can be
> > probably put in an rcu protected array? I'm not sure what's the
> > synchronization convention around node on/offlining. Is that protected
> > together with CPU on/offlining?
> 
> The machinery is already there, we just need another array of nodemasks - 
> sched_domains_numa_nodes in addition to sched_domains_numa_nodes. The
> last one is already protected by RCU, and we need to update new array every
> time when sched_domains_numa_nodes updated.
>  
> > Given that there usually aren't that many nodes, the current implementation
> > is probably fine too, so please feel free to ignore this suggestion for now
> > too.
> 
> I agree. The number of nodes on typical system is 1 or 2. Even if
> it's 8, the Andrea's bubble sort will be still acceptable. So, I'm
> OK with O(N^2) if you guys OK with it. I only would like to have
> this choice explained in commit message.

Good point, I'll add a comment about that.

Thanks,
-Andrea