[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <SJ1PR11MB60836CC0C71BB44059B72C49FC4F2@SJ1PR11MB6083.namprd11.prod.outlook.com>
Date: Tue, 13 Feb 2024 19:02:22 +0000
From: "Luck, Tony" <tony.luck@...el.com>
To: James Morse <james.morse@....com>, "Chatre, Reinette"
<reinette.chatre@...el.com>
CC: "babu.moger@....com" <babu.moger@....com>, "Yu, Fenghua"
<fenghua.yu@...el.com>, Peter Newman <peternewman@...gle.com>, "Jonathan
Corbet" <corbet@....net>, Shuah Khan <skhan@...uxfoundation.org>,
"x86@...nel.org" <x86@...nel.org>, Shaopeng Tan <tan.shaopeng@...itsu.com>,
Jamie Iles <quic_jiles@...cinc.com>, Randy Dunlap <rdunlap@...radead.org>,
Drew Fustini <dfustini@...libre.com>, "linux-kernel@...r.kernel.org"
<linux-kernel@...r.kernel.org>, "linux-doc@...r.kernel.org"
<linux-doc@...r.kernel.org>, "patches@...ts.linux.dev"
<patches@...ts.linux.dev>
Subject: RE: [PATCH v15-RFC 0/8] Add support for Sub-NUMA cluster (SNC)
systems
> With SNC enable, the L3 monitors are unaffected, but the controls behave as if they were
> part of some other component in the system.
I don't think of it like that. See attached picture of a single socket divided in two by SNC.
[If the attachment is stripped off for those reading this via mailing lists, if you want the
picture, just send me an e-mail.]
Everything in blue is node 0. Yellow for node 1.
The rectangles in the middle represent the L3 cache (12-way associative). When cores
in node 0 access memory in node 0, it will be cached using the "top" half of the cache
indices. Similarly for node 1 using the "bottom" half.
Here’s how each of the Intel L3 resctrl functions operate with SNC enabled:
CQM: Reports how much of your half of the L3 cache is occupied
MBM: Reports on memory traffic from your half of the cache to your memory controllers.
CAT: Still controls which ways of the cache are available for allocation (but each way
has half the capacity.)
MBA: The same throttling levels applied to "blue" and "yellow" traffic (because there
are only socket level controls).
> I'm a little nervous that the SNC support looks strange if we ever add support for
> something like the above. Given its described in ACPI, I assume there are plenty of
> machines out there that look like this.
I'm also nervous as h/w designers find various ways to diverge from the old paradigm of
socket scope == L3 cache scope == NUMA node scope
-Tony
Download attachment "SNC topology.png" of type "image/png" (18304 bytes)
Powered by blists - more mailing lists