linux-kernel - Re: [PATCH v7 00/10] x86,fs/resctrl: Support L3 Smart Data Cache Injection Allocation Enforcement (SDCIAE)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <67f1cc75-f678-4112-bd2d-f4f3abcdd845@amd.com>
Date: Mon, 4 Aug 2025 16:34:04 -0500
From: "Moger, Babu" <babu.moger@....com>
To: Reinette Chatre <reinette.chatre@...el.com>, corbet@....net,
 tony.luck@...el.com, Dave.Martin@....com, james.morse@....com,
 tglx@...utronix.de, mingo@...hat.com, bp@...en8.de,
 dave.hansen@...ux.intel.com
Cc: x86@...nel.org, hpa@...or.com, akpm@...ux-foundation.org,
 paulmck@...nel.org, rostedt@...dmis.org, Neeraj.Upadhyay@....com,
 david@...hat.com, arnd@...db.de, fvdl@...gle.com, seanjc@...gle.com,
 thomas.lendacky@....com, pawan.kumar.gupta@...ux.intel.com,
 yosry.ahmed@...ux.dev, sohil.mehta@...el.com, xin@...or.com,
 kai.huang@...el.com, xiaoyao.li@...el.com, peterz@...radead.org,
 me@...aill.net, mario.limonciello@....com, xin3.li@...el.com,
 ebiggers@...gle.com, ak@...ux.intel.com, chang.seok.bae@...el.com,
 andrew.cooper3@...rix.com, perry.yuan@....com, linux-doc@...r.kernel.org,
 linux-kernel@...r.kernel.org
Subject: Re: [PATCH v7 00/10] x86,fs/resctrl: Support L3 Smart Data Cache
 Injection Allocation Enforcement (SDCIAE)

Hi Reinette,


On 7/21/25 18:28, Reinette Chatre wrote:
> Hi Babu,
> 
> On 7/10/25 10:16 AM, Babu Moger wrote:
>>
>> This series adds the support for L3 Smart Data Cache Injection Allocation
>> Enforcement (SDCIAE) to resctrl infrastructure. It is referred to as
>> "io_alloc" in resctrl subsystem.
>>
>> Upcoming AMD hardware implements Smart Data Cache Injection (SDCI).
>> Smart Data Cache Injection (SDCI) is a mechanism that enables direct
>> insertion of data from I/O devices into the L3 cache. By directly caching
>> data from I/O devices rather than first storing the I/O data in DRAM, SDCI
>> reduces demands on DRAM bandwidth and reduces latency to the processor
>> consuming the I/O data.
>>
>> The SDCIAE (SDCI Allocation Enforcement) PQE feature allows system software
>> to control the portion of the L3 cache used for SDCI devices.
>>
>> When enabled, SDCIAE forces all SDCI lines to be placed into the L3 cache
>> partitions identified by the highest-supported L3_MASK_n register, where n
>> is the maximum supported CLOSID.
> 
> Even though this CLOSID use is unique to AMD this implementation makes it part
> of resctrl fs's support of io_alloc. It is confusing to have architectural
> specific features be handled by resctrl fs so I think it will be useful to
> add a snippet here to help folks trying to decipher this work. Consider
> for example a snippet like:
> 
>   Since CLOSIDs are managed by resctrl fs it is least invasive to make
>   the "io_alloc is supported by maximum supported CLOSID" part of the
>   initial resctrl fs support for io_alloc. Take care not to expose this
>   use of CLOSID for io_alloc to user space so that this is not required from
>   other architectures that may support io_alloc differently in the future.
> 

Sure.

>>
>> The feature details are documented in the APM listed below [1].
>> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
>> Publication # 24593 Revision 3.41 section 19.4.7 L3 Smart Data Cache
>> Injection Allocation Enforcement (SDCIAE)
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
>>
>> The feature requires linux support of TPH (TLP Processing Hints).
>> The support is available in linux kernel after the commit
>> 48d0fd2b903e3 ("PCI/TPH: Add TPH documentation")
>>
>> The patches are based on top of commit (6.16.0-rc5)
>> commit b4ec95e3bc3f ("Merge x86/microcode into tip/master")
>>
>> # Linux Implementation
>>
>> Feature adds following interface files when the resctrl "io_alloc" feature
>> is supported on the resource:
>>
>> /sys/fs/resctrl/info/L3/io_alloc: Report the feature status. Enable/disable the
>> 				  feature by writing to the interface.
>>
>> /sys/fs/resctrl/info/L3/io_alloc_cbm:  List the Capacity Bit Masks (CBMs) available
>> 				       for I/O devices when io_alloc feature is enabled.
>> 				       Configure the CBM by writing to the interface.
>>
>> When CDP is enabled, these files will be created both in L3CODE and L3DATA.
> 
> "will be" -> "are"
> 

Sure.

>>
>> # Examples:
>>
>> a. Check if io_alloc feature is available
>> 	# mount -t resctrl resctrl /sys/fs/resctrl/
>>
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc
>> 	disabled
>>
>> b. Enable the io_alloc feature. 
>>
>> 	# echo 1 > /sys/fs/resctrl/info/L3/io_alloc 
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc
>> 	enabled
>>
>> c. Check the CBM values for the io_alloc feature.
>>
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc_cbm 
>> 	0=ffff;1=ffff
>>
>> d. Change the CBM value for the domain 1:
> 
> "for the domain 1" -> "of domain 1"?

Sure.

> 
> (nit: inconsistent use of "." vs. ":" in items)

Fixed it.

> 
>> 	# echo 1=FF > /sys/fs/resctrl/info/L3/io_alloc_cbm
>>
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc_cbm 
>> 	0=ffff;1=00ff
>>
>> d. Disable io_alloc feature and exit.
>>
>> 	# echo 0 > /sys/fs/resctrl/info/L3/io_alloc 
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc
>> 	disabled
>>
>> 	# umount /sys/fs/resctrl/
>>
>> ---
> 
> Reinette
> 
> 

-- 
Thanks
Babu Moger