linux-kernel - RE: [PATCH v15-RFC 0/8] Add support for Sub-NUMA cluster (SNC) systems

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date: Tue, 13 Feb 2024 19:02:22 +0000
From: "Luck, Tony" <tony.luck@...el.com>
To: James Morse <james.morse@....com>, "Chatre, Reinette"
	<reinette.chatre@...el.com>
CC: "babu.moger@....com" <babu.moger@....com>, "Yu, Fenghua"
	<fenghua.yu@...el.com>, Peter Newman <peternewman@...gle.com>, "Jonathan
 Corbet" <corbet@....net>, Shuah Khan <skhan@...uxfoundation.org>,
	"x86@...nel.org" <x86@...nel.org>, Shaopeng Tan <tan.shaopeng@...itsu.com>,
	Jamie Iles <quic_jiles@...cinc.com>, Randy Dunlap <rdunlap@...radead.org>,
	Drew Fustini <dfustini@...libre.com>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "linux-doc@...r.kernel.org"
	<linux-doc@...r.kernel.org>, "patches@...ts.linux.dev"
	<patches@...ts.linux.dev>
Subject: RE: [PATCH v15-RFC 0/8] Add support for Sub-NUMA cluster (SNC)
 systems

> With SNC enable, the L3 monitors are unaffected, but the controls behave as if they were
> part of some other component in the system.

I don't think of it like that. See attached picture of a single socket divided in two by SNC.
[If the attachment is stripped off for those reading this via mailing lists, if you want the
picture, just send me an e-mail.]

Everything in blue is node 0. Yellow for node 1.

The rectangles in the middle represent the L3 cache (12-way associative). When cores
in node 0 access memory in node 0, it will be cached using the "top" half of the cache
indices. Similarly for node 1 using the "bottom" half.

Here’s how each of the Intel L3 resctrl functions operate with SNC enabled:

CQM: Reports how much of your half of the L3 cache is occupied

MBM: Reports on memory traffic from your half of the cache to your memory controllers.

CAT: Still controls which ways of the cache are available for allocation (but each way
has half the capacity.)

MBA: The same throttling levels applied to "blue" and "yellow" traffic (because there
are only socket level controls).

> I'm a little nervous that the SNC support looks strange if we ever add support for
> something like the above. Given its described in ACPI, I assume there are plenty of
> machines out there that look like this.

I'm also nervous as h/w designers find various ways to diverge from the old paradigm of

	socket scope == L3 cache scope == NUMA node scope

-Tony

Download attachment "SNC topology.png" of type "image/png" (18304 bytes)