linux-kernel - Re: [PATCH] sched: Make select_idle_sibling search domain configurable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200728103907.GT119549@hirez.programming.kicks-ass.net>
Date:   Tue, 28 Jul 2020 12:39:07 +0200
From:   peterz@...radead.org
To:     Xi Wang <xii@...gle.com>
Cc:     Ingo Molnar <mingo@...hat.com>, Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Mel Gorman <mgorman@...e.de>, linux-kernel@...r.kernel.org,
        linux-fsdevel@...r.kernel.org, suravee.suthikulpanit@....com,
        thomas.lendacky@....com
Subject: Re: [PATCH] sched: Make select_idle_sibling search domain
 configurable

On Tue, Jul 28, 2020 at 12:01:31AM -0700, Xi Wang wrote:
> The scope of select_idle_sibling idle cpu search is LLC. This
> becomes a problem for the AMD CCX architecture, as the sd_llc is only
> 4 cores. On a many core machine, the range of search is too small to
> reach a satisfactory level of statistical multiplexing / efficient
> utilization of short idle time slices.
> 
> With this patch idle sibling search is detached from LLC and it
> becomes run time configurable. To reduce search and migration
> overheads, a presearch domain is added. The presearch domain will be
> searched first before the "main search" domain, e.g.:
> 
> sysctl_sched_wake_idle_domain == 2 ("MC" domain)
> sysctl_sched_wake_idle_presearch_domain == 1 ("DIE" domain)
> 
> Presearch will go through 4 cores of a CCX. If no idle cpu is found
> during presearch, full search will go through the remaining cores of
> a cpu socket.

*groan*, this is horrific :-(

It is also in direct conflict with people wanting to make it smaller.

On top of that, a domain number is a terrible terrible interface. They
aren't even available without SCHED_DEBUG on.

What is the inter-L3 latency? Going by this that had better be awesome.
And if this Infinity Fabric stuff if highly effective in effectively
merging L3s -- analogous to what Intel does with it's cache slices, then
should we not change the AMD topology setup instead of this 'thing'?

Also, this commit:

  051f3ca02e46 ("sched/topology: Introduce NUMA identity node sched domain")

seems to suggest L3 is actually bigger. Suravee, can you please comment?