linux-kernel - Re: [PATCH sched_ext/for-6.13] sched_ext: Do not enable LLC/NUMA optimizations when domains overlap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Zyq4VFpbaKXERdDh@gpd3>
Date: Wed, 6 Nov 2024 01:29:08 +0100
From: Andrea Righi <arighi@...dia.com>
To: Tejun Heo <tj@...nel.org>
Cc: David Vernet <void@...ifault.com>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH sched_ext/for-6.13] sched_ext: Do not enable LLC/NUMA
 optimizations when domains overlap

On Tue, Nov 05, 2024 at 01:32:05PM -1000, Tejun Heo wrote:
> Hello, Andrea.
> 
> Sorry about the delay.
> 
> On Thu, Oct 31, 2024 at 08:34:01AM +0100, Andrea Righi wrote:
> > @@ -3154,16 +3154,29 @@ static void update_selcpu_topology(void)
> >       rcu_read_lock();
> >       sd = rcu_dereference(per_cpu(sd_llc, cpu));
> >       if (sd) {
> > -             cpus = sched_domain_span(sd);
> > -             if (cpumask_weight(cpus) < num_possible_cpus())
> > +             llc_cpus = sched_domain_span(sd);
> > +             if (cpumask_weight(llc_cpus) < num_possible_cpus())
> 
> Not from this patch but should the weight be compared against
> num_online_cpus()? Sched domains don't include offline CPUs, right?

That's right, sched domain is definitely updated when a CPU goes offine
(I've just verified), so we should check for num_online_cpus() here.
Thanks for noticing it!

> 
> ...
> > +     /*
> > +      * If the NUMA domain perfectly overlaps with the LLC domain, enable
> > +      * LLC optimization only, as checking for an idle CPU in the same
> > +      * domain twice is redundant.
> > +      */
> > +     if (enable_numa && enable_llc && cpumask_equal(numa_cpus, llc_cpus))
> > +             enable_numa = false;
> > +     /*
> > +      * If all the online CPUs are in the same LLC domain, there is no
> > +      * reason to enable LLC optimization.
> > +      */
> > +     if (enable_llc && cpumask_equal(llc_cpus, cpu_online_mask))
> > +             enable_llc = false;
> 
> The second test looks like it should always be correct. I'm not sure the
> first one is. It probably is good enough but would at least be worthwhile to
> explain why that is?

Let's say we have 2 NUMA nodes, each with 2 sockets, and each socket
has its own L3 cache. In this case, numa_cpus will be larger than
llc_cpus, and enabling both NUMA and LLC optimizations would be
beneficial.

On the other hand, if each NUMA node contains only 1 socket, numa_cpus
and llc_cpus will overlap completely, making it unnecessary to enable
both NUMA and LLC optimizations, so we can have just the LLC in this
case.

Would something like this help clarifying the first test?

Thanks,
-Andrea