| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <3fddbb5b-a6ac-4465-afc0-6365225ba2bb@intel.com>
Date: Fri, 21 Nov 2025 08:05:36 -0800
From: Reinette Chatre <reinette.chatre@...el.com>
To: Babu Moger <babu.moger@....com>, <tony.luck@...el.com>,
<tglx@...utronix.de>, <mingo@...hat.com>, <bp@...en8.de>,
<dave.hansen@...ux.intel.com>, <x86@...nel.org>
CC: <corbet@....net>, <Dave.Martin@....com>, <james.morse@....com>,
<hpa@...or.com>, <akpm@...ux-foundation.org>, <paulmck@...nel.org>,
<rdunlap@...radead.org>, <pmladek@...e.com>, <kees@...nel.org>,
<arnd@...db.de>, <fvdl@...gle.com>, <seanjc@...gle.com>,
<pawan.kumar.gupta@...ux.intel.com>, <xin@...or.com>,
<thomas.lendacky@....com>, <sohil.mehta@...el.com>, <jarkko@...nel.org>,
<chang.seok.bae@...el.com>, <ebiggers@...gle.com>,
<elena.reshetova@...el.com>, <ak@...ux.intel.com>,
<mario.limonciello@....com>, <perry.yuan@....com>,
<linux-doc@...r.kernel.org>, <linux-kernel@...r.kernel.org>,
<peternewman@...gle.com>, <feng.tang@...ux.alibaba.com>
Subject: Re: [PATCH v12 00/10] x86,fs/resctrl: Support L3 Smart Data Cache
Injection Allocation Enforcement (SDCIAE)
Dear x86 maintainers,
Could you please consider this series for inclusion when you find is most appropriate?
Regards,
Reinette
On 11/12/25 4:57 PM, Babu Moger wrote:
>
> This series adds the support for L3 Smart Data Cache Injection Allocation
> Enforcement (SDCIAE) to resctrl infrastructure. It is referred to as
> "io_alloc" in resctrl subsystem.
>
> Upcoming AMD hardware implements Smart Data Cache Injection (SDCI).
> Smart Data Cache Injection (SDCI) is a mechanism that enables direct
> insertion of data from I/O devices into the L3 cache. By directly caching
> data from I/O devices rather than first storing the I/O data in DRAM, SDCI
> reduces demands on DRAM bandwidth and reduces latency to the processor
> consuming the I/O data.
>
> The SDCIAE (SDCI Allocation Enforcement) PQE feature allows system software
> to control the portion of the L3 cache used for SDCI devices.
>
> When enabled, SDCIAE forces all SDCI lines to be placed into the L3 cache
> partitions identified by the highest-supported L3_MASK_n register, where n
> is the maximum supported CLOSID.
>
> Since CLOSIDs are managed by resctrl fs it is least invasive to make
> the "io_alloc is supported by maximum supported CLOSID" part of the
> initial resctrl fs support for io_alloc. Take care not to expose this
> use of CLOSID for io_alloc to user space so that this is not required from
> other architectures that may support io_alloc differently in the future.
>
> The SDCIAE feature details are documented in APM [1] available from [2].
> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
> Publication # 24593 Revision 3.41 section 19.4.7 L3 Smart Data Cache
> Injection Allocation Enforcement (SDCIAE)
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
>
> The feature requires linux support of TPH (TLP Processing Hints).
> The support is available in linux kernel after the commit
> 48d0fd2b903e3 ("PCI/TPH: Add TPH documentation")
>
> The patches are based on top of commit (v6.18.0-rc5)
> Commit 2f70cdbd7c20 (tip/master) ("Merge branch into tip/master: 'x86/sgx'")
>
> Comments and suggestions are always welcome as usual.
>
> # Linux Implementation
>
> Feature adds following interface files when the resctrl "io_alloc" feature
> is supported on the resource:
>
> /sys/fs/resctrl/info/L3/io_alloc: Report the feature status. Enable/disable the
> feature by writing to the interface.
>
> /sys/fs/resctrl/info/L3/io_alloc_cbm: List the Capacity Bit Masks (CBMs) available
> for I/O devices when io_alloc feature is enabled.
> Configure the CBM by writing to the interface.
>
> When CDP is enabled, these files are created both in L3CODE and L3DATA.
>
> # Examples:
>
> a. Check if io_alloc feature is available.
>
> # mount -t resctrl resctrl /sys/fs/resctrl/
>
> # cat /sys/fs/resctrl/info/L3/io_alloc
> disabled
>
> b. Enable the io_alloc feature.
>
> # echo 1 > /sys/fs/resctrl/info/L3/io_alloc
> # cat /sys/fs/resctrl/info/L3/io_alloc
> enabled
>
> c. Check the CBM values for the io_alloc feature.
>
> # cat /sys/fs/resctrl/info/L3/io_alloc_cbm
> 0=ffff;1=ffff
>
> d. Change the CBM value of domain 1.
> # echo 1=ff > /sys/fs/resctrl/info/L3/io_alloc_cbm
>
> # cat /sys/fs/resctrl/info/L3/io_alloc_cbm
> 0=ffff;1=00ff
>
> e. Change the CBM value of domain 0 and 1.
> # echo "0=ff;1=f" > /sys/fs/resctrl/info/L3/io_alloc_cbm
>
> # cat /sys/fs/resctrl/info/L3/io_alloc_cbm
> 0=00ff;1=000f
>
>
> f. Disable io_alloc feature and exit.
>
> # echo 0 > /sys/fs/resctrl/info/L3/io_alloc
> # cat /sys/fs/resctrl/info/L3/io_alloc
> disabled
>
> # umount /sys/fs/resctrl/
> ---
> v12:
> Minor format fix in resctrl.rst (patch 6) and added text about writable files in the info section.
> Other patches did not change.
> There were some discussion about adding support to update all the domains of io_alloc_cbm
> with one command.
> https://lore.kernel.org/lkml/20251107012401.224515-2-atomlin@atomlin.com/
> Aaron will be adding that later.
>
> v11:
> Took care of review commnet on patch 6.
> Fixed typo CDP_3DATA -> CDP_DATA.
> Added code to free the CLOSID on resctrl_arch_io_alloc_enable() failure.
> Fixed the prototype of rdtgroup_name_by_closid().
>
> Other patches did not change.
> Added Reviewed-by: tag for few patches.
> Took care of few minor conflicts in fs/resctrl/internal.h due to code displacement.
>
> v10:
> Updated the changelogs in most patches when there is some repetitions in texts. Thanks to Reinette.
>
> Fixed the minor conflicts in scattered.c and cpufeatures.h.
>
> Fixed the few conflicts due to recent ABMC feature merge.
>
> Code comment update to match MSR names correctly.
>
> Re-arranged the details in resctrl.rst file.
>
> Changed the text L3CODE to CDP_CODE and L3DATA to CDP_DATA.
>
> Updated the return code to EINVAL when feature is not enabled in resctrl_io_alloc_cbm_write().
>
> Few other text changes. Added details in each patch about the specifics.
>
> v9:
> Major change is related to CDP.
> The processing and updating CBMs for CDP_CODE and CDP_DATA are done only once.
> The updated CBMs are copied to the peers using staged_config.
>
> Removed resctrl_get_schema(). Not required anymore.
>
> Updated the "bit_usage" section of resctrl.rst for io_alloc.
>
> Fixed the tabs in SMBA and BMEC lines in resctrl.rst.
>
> Improved the changelog for all the patches.
>
> Added the code comments about CDP_CODE and CDP_DATA where applicable.
>
> Added Reviewed-by: tag for couple of patches.
>
> Added comments in each patch about the changes.
>
>
> v8:
> Added Acked-by, Reviewed-by tags to couple of patches.
>
> Updated Documentation/filesystems/resctrl.rst for each interface.
>
> Updated the changelog in most patches.
>
> Moved the patch to update the rdt_bit_usage_show() for io_alloc changes to the end.
>
> Moved resctrl_arch_io_alloc_enable() and its dependancies to
> arch/x86/kernel/cpu/resctrl/ctrlmondata.c file.
>
> Moved resctrl_io_alloc_show() to fs/resctrl/ctrlmondata.c.
> Added prototype for rdt_kn_parent_priv() in fs/resctrl/internal.h
> so, it can be uses in other fs/resctrl/ files.
>
> Renamed resctrl_io_alloc_init_cat() to resctrl_io_alloc_init_cbm().
> Moved resctrl_io_alloc_write() and all its dependancies to fs/resctrl/ctrlmondata.c.
> Added prototypes for all the functions in fs/resctrl/internal.h.
>
> Moved resctrl_io_alloc_cbm_show() to fs/resctrl/ctrlmondata.c. show_doms remains
> static with this change.
>
> Moved resctrl_io_alloc_parse_line() and resctrl_io_alloc_cbm_write()
> to fs/resctrl/ctrlmondata.c.
>
> Added resctrl_arch_get_cdp_enabled() check inside resctrl_io_alloc_parse_line().
> parse_cbm() remains static as everything moved to ctrlmondata.c.
>
> Simplified the CDT check in rdt_bit_usage_show() as CDP_DATA and CDP_CODE
> are in sync with io_alloc enabled.
>
> v7:
> Fixed few conflicts in
> arch/x86/include/asm/cpufeatures.h
> arch/x86/kernel/cpu/scattered.c
>
> Updated the changelog in most patches. Removed the references of L3 in
> filesystem related changes.
>
> Removed the inline for resctrl_arch_get_io_alloc_enabled().
> Updated the code comment in resctrl.h.
> Changed the subject to x86,fs/resctrl where applicable.
>
> Split the patches based on the comment.
> https://lore.kernel.org/lkml/3bec3844-7fda-452b-988f-42b0de9d63ba@intel.com/
> Separated resctrl_io_alloc_show and bit_usage changes in two separate patches.
>
> Added new function resctrl_io_alloc_closid_supported() to verify io_alloc CLOSID.
>
> Added the code to initialize/update the schemata for both CDP_DATA and CDP_CODE when CDP is enabled.
>
> Rephrased the changelog and code comments in all the patches.
>
> v6:
> Sorry if you see this series duplicate. Messed up the
> emails linux-doc@...r.kernel.org and linux-kernel@...r.kernel.org.
>
> Sent v5 by mistake before completing all testing.
> Most of the changes are in resctrl.rst user doc.
> The resource name is no longer printed in io_alloc_cbms.
> Updated the related documentation accordingly.
> Resolved conflicts in cpufeatures.h
> Added lockdep_assert_cpus_held() in _resctrl_sdciae_enable() to protect
> r->ctrl_domains.
>
> Added more comments in include/linux/resctrl.h.
>
> Updated "io_alloc_cbm" details in user doc resctrl.rst. Resource name is
> not printed in CBM now.
>
> Updated subjects to fs/resctrl: where applicable.
>
> v5:
> Patches are created on top of recent resctrl FS/ARCH code restructure.
> The files monitor.c/rdtgroup.c have been split between FS and ARCH directories.
> Resolved the conflict due to the merge.
>
> Updated bit_usage to reflect the io_alloc CBM as discussed in the thread:
> https://lore.kernel.org/lkml/3ca0a5dc-ad9c-4767-9011-b79d986e1e8d@intel.com/
> Modified rdt_bit_usage_show() to read io_alloc_cbm in hw_shareable, ensuring
> that bit_usage accurately represents the CBMs.
>
> Moved prototypes of resctrl_arch_io_alloc_enable() and
> resctrl_arch_get_io_alloc_enabled() to include/linux/resctrl.h.
>
> Used rdt_kn_name to get the rdtgroup name instead of accesssing it directly
> while printing group name used by the io_alloc_closid.
>
> Updated show_doms() to print the resource if only it is valid. Pass NULL while
> printing io_alloc CBM.
>
> Changed the code to access io_alloc CBMs via either L3CODE or L3DATA resources.
>
> v4: The "io_alloc" interface will report "enabled/disabled/not supported"
> instead of 0 or 1..
>
> Updated resctrl_io_alloc_closid_get() to verify the max closid availability
> using closids_supported().
>
> Updated the documentation for "shareable_bits" and "bit_usage".
>
> NOTE: io_alloc is about specific CLOS. rdt_bit_usage_show() is not designed
> handle bit_usage for specific CLOS. Its about overall system. So, we cannot
> really tell the user which CLOS is shared across both hardware and software.
> This is something we need to discuss.
>
> Introduced io_alloc_init() to initialize fflags.
>
> Printed the group name when io_alloc enablement fails to help user.
>
> Added rdtgroup_mutex before rdt_last_cmd_puts() in resctrl_io_alloc_cbm_show().
> Returned -ENODEV when resource type is CDP_DATA.
>
> Kept the resource name while printing the CBM (L3:0=ffff) that way we dont have
> to change show_doms() just for this feature and it is consistant across all the
> schemata display.
>
> Added new patch to call parse_cbm() directly to avoid code duplication.
>
> Checked all the series(v1-v3) again to verify if I missed any comment.
>
> v3: Rewrote commit log for the last 3 patches. Changed the text to bit
> more generic than the AMD specific feature. Added AMD feature
> specifics in the end.
>
> Renamed the rdt_get_sdciae_alloc_cfg() to rdt_set_io_alloc_capable().
> Renamed the _resctrl_io_alloc_enable() to _resctrl_sdciae_enable()
> as it is arch specific.
>
> Changed the return to void in _resctrl_sdciae_enable() instead of int.
>
> The number of CLOSIDs is determined based on the minimum supported
> across all resources (in closid_init). It needs to match the max
> supported on the resource. Added the check to verify if MAX CLOSID
> availability on the system.
>
> Added CDP check to make sure io_alloc is configured in CDP_CODE.
> Highest CLOSID corresponds to CDP_CODE.
>
> Added resctrl_io_alloc_closid_free() to free the io_alloc CLOSID.
>
> Added errors in few cases when CLOSID allocation fails.
> Fixes splat reported when info/L3/bit_usage is accesed when io_alloc is enabled.
> https://lore.kernel.org/lkml/SJ1PR11MB60837B532254E7B23BC27E84FC052@SJ1PR11MB6083.namprd11.prod.outlook.com/
>
> v2: Added dependancy on X86_FEATURE_CAT_L3
> Removed the "" in CPU feature definition.
>
> Changed sdciae_capable to io_alloc_capable to make it as generic feature.
> Moved io_alloc_capable field in struct resctrl_cache.
>
> Changed the name of few arch functions similar to ABMC series.
> resctrl_arch_get_io_alloc_enabled()
> resctrl_arch_io_alloc_enable()
>
> Renamed the feature to "io_alloc".
> Added generic texts for the feature in commit log and resctrl.rst doc.
> Added resctrl_io_alloc_init_cat() to initialize io_alloc to default values
> when enabled.
> Fixed io_alloc interface to show only on L3 resource.
> Added the locks while processing io_alloc CBMs.
>
> Previous versions:
> v11: https://lore.kernel.org/lkml/cover.1761844489.git.babu.moger@amd.com/
> v10: https://lore.kernel.org/lkml/cover.1761090859.git.babu.moger@amd.com/
> v9: https://lore.kernel.org/lkml/cover.1756851697.git.babu.moger@amd.com/
> v8: https://lore.kernel.org/lkml/cover.1754436586.git.babu.moger@amd.com/
> v7: https://lore.kernel.org/lkml/cover.1752167718.git.babu.moger@amd.com/
> v6: https://lore.kernel.org/lkml/cover.1749677012.git.babu.moger@amd.com/
> v5: https://lore.kernel.org/lkml/cover.1747943499.git.babu.moger@amd.com/
> v4: https://lore.kernel.org/lkml/cover.1745275431.git.babu.moger@amd.com/
> v3: https://lore.kernel.org/lkml/cover.1738272037.git.babu.moger@amd.com/
> v2: https://lore.kernel.org/lkml/cover.1734556832.git.babu.moger@amd.com/
> v1: https://lore.kernel.org/lkml/cover.1723824984.git.babu.moger@amd.com/
>
>
> Babu Moger (10):
> x86/cpufeatures: Add support for L3 Smart Data Cache Injection
> Allocation Enforcement
> x86/resctrl: Add SDCIAE feature in the command line options
> x86,fs/resctrl: Detect io_alloc feature
> x86,fs/resctrl: Implement "io_alloc" enable/disable handlers
> fs/resctrl: Introduce interface to display "io_alloc" support
> fs/resctrl: Add user interface to enable/disable io_alloc feature
> fs/resctrl: Introduce interface to display io_alloc CBMs
> fs/resctrl: Modify struct rdt_parse_data to pass mode and CLOSID
> fs/resctrl: Introduce interface to modify io_alloc capacity bitmasks
> fs/resctrl: Update bit_usage to reflect io_alloc
>
> .../admin-guide/kernel-parameters.txt | 2 +-
> Documentation/filesystems/resctrl.rst | 134 ++++++--
> arch/x86/include/asm/cpufeatures.h | 1 +
> arch/x86/kernel/cpu/cpuid-deps.c | 1 +
> arch/x86/kernel/cpu/resctrl/core.c | 9 +
> arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 40 +++
> arch/x86/kernel/cpu/resctrl/internal.h | 5 +
> arch/x86/kernel/cpu/scattered.c | 1 +
> fs/resctrl/ctrlmondata.c | 309 +++++++++++++++++-
> fs/resctrl/internal.h | 17 +
> fs/resctrl/rdtgroup.c | 75 ++++-
> include/linux/resctrl.h | 24 ++
> 12 files changed, 574 insertions(+), 44 deletions(-)
>
Powered by blists - more mailing lists