Message-ID: <cae62055-f244-4c3e-9d68-6e86b5e221b7@amd.com>
Date: Tue, 17 Sep 2024 15:51:54 -0500
From: "Moger, Babu" <babu.moger@....com>
To: Reinette Chatre <reinette.chatre@...el.com>, corbet@....net,
 tglx@...utronix.de, mingo@...hat.com, bp@...en8.de,
 dave.hansen@...ux.intel.com, x86@...nel.org
Cc: fenghua.yu@...el.com, hpa@...or.com, paulmck@...nel.org,
 thuth@...hat.com, xiongwei.song@...driver.com, ardb@...nel.org,
 pawan.kumar.gupta@...ux.intel.com, daniel.sneddon@...ux.intel.com,
 sandipan.das@....com, kai.huang@...el.com, peterz@...radead.org,
 kan.liang@...ux.intel.com, pbonzini@...hat.com, xin3.li@...el.com,
 ebiggers@...gle.com, alexandre.chartre@...cle.com, perry.yuan@....com,
 tan.shaopeng@...itsu.com, james.morse@....com, tony.luck@...el.com,
 maciej.wieczor-retman@...el.com, linux-doc@...r.kernel.org,
 linux-kernel@...r.kernel.org, peternewman@...gle.com, eranian@...gle.com
Subject: Re: [PATCH 0/7] x86/resctrl : Support L3 Smart Data Cache Injection
 Allocation Enforcement (SDCIAE)

Hi Reinette,

Thanks for the feedback.

On 9/13/24 15:44, Reinette Chatre wrote:
> Hi Babu,
> 
> On 8/16/24 9:16 AM, Babu Moger wrote:
>>
>> This series adds the support for L3 Smart Data Cache Injection Allocation
>> Enforcement (SDCIAE) to resctrl infrastructure.
>>
>> Upcoming AMD hardware implements Smart Data Cache Injection (SDCI).
>> Smart Data Cache Injection (SDCI) is a mechanism that enables direct
>> insertion of data from I/O devices into the L3 cache. By directly caching
>> data from I/O devices rather than first storing the I/O data in DRAM, SDCI
>> reduces demands on DRAM bandwidth and reduces latency to the processor
>> consuming the I/O data. The SDCIAE (SDCI Allocation Enforcement) PQE
>> feature allows system software to limit the portion of the L3 cache used
>> for SDCI.
>>
> 
> This series introduces new user interface. Could you please describe the
> new user interface in the cover letter and how users are expected to interact
> with this interface to be able to use this new feature? Please also describe
> the impact on existing resctrl interfaces related to cache allocation from
> I/O hardware, for example "shareable_bits", "bit_usage", etc. These existing
> interfaces are used to communicate to user space how portions of cache are
> used by I/O hardware but I cannot tell from this series how this work
> builds on
> this.

Sure. Will add interface details and bit mask usage.

> 
> How does this feature work with the existing "L3 Cache Allocation Sharing
> Mask"
> that is enumerated as part of CAT feature?

Sharing is allowed as it is right now. SDCIAE cannot use the masks that
are exclusive.

> 
>> The feature details are documented in the APM listed below [1].
>> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
>> Publication # 24593 Revision 3.41 section 19.4.7 L3 Smart Data Cache
>> Injection Allocation Enforcement (SDCIAE)
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
>>
>> The feature requires linux support of TPH (TLP Processing Hints).
>> The support is ongoing and patches are currently under review.
>> https://lore.kernel.org/lkml/20240717205511.2541693-2-wei.huang2@amd.com/
> 
> Please note that the cover letter [1] of that series mentions "Cache
> Injection
> allows PCIe endpoints to inject I/O Coherent DMA writes directly into an
> L2 ..."
> while this series implements and refers to L3 only.

SDCIAE only applies to L3 insertion. SDCIAE has no impact on SDCI
insertion into the L2.

SDCI lines will be inserted into the L3 after being evicted by L2 based on
the maximum CLOS(CLOSIS 15) allocation mask when SDCIAE is enabled. By
limiting the number of ways in L3, it will impact the SDCI cache occupancy
in L3.
-- 
Thanks
Babu Moger

Powered by blists - more mailing lists