Message-ID: <CALPaoCh=brY6+_oJZXJLfpv_Qkz0i0RuPfuRa+TgXLu_xV6rsQ@mail.gmail.com>
Date: Thu, 15 Jan 2026 12:14:19 +0100
From: Peter Newman <peternewman@...gle.com>
To: Ben Horgan <ben.horgan@....com>
Cc: amitsinght@...vell.com, baisheng.gao@...soc.com, 
	baolin.wang@...ux.alibaba.com, carl@...amperecomputing.com, 
	dave.martin@....com, david@...nel.org, dfustini@...libre.com, 
	fenghuay@...dia.com, gshan@...hat.com, james.morse@....com, 
	jonathan.cameron@...wei.com, kobak@...dia.com, lcherian@...vell.com, 
	linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org, 
	punit.agrawal@....qualcomm.com, quic_jiles@...cinc.com, 
	reinette.chatre@...el.com, rohit.mathew@....com, scott@...amperecomputing.com, 
	sdonthineni@...dia.com, tan.shaopeng@...itsu.com, xhao@...ux.alibaba.com, 
	catalin.marinas@....com, will@...nel.org, corbet@....net, maz@...nel.org, 
	oupton@...nel.org, joey.gouly@....com, suzuki.poulose@....com, 
	kvmarm@...ts.linux.dev
Subject: Re: [PATCH v3 00/47] arm_mpam: Add KVM/arm64 and resctrl glue code

Hi Ben,

On Mon, Jan 12, 2026 at 5:59 PM Ben Horgan <ben.horgan@....com> wrote:
>
> This new version of the mpam missing pieces has no major rework from the
> previous version. It's mainly small corrections and code tidying based on
> review and things I spotted along the way. To be able to merge this we need
> review from more people and people to start testing on their platforms and
> giving some Tested-by tags.
>
> Change list in patches.
>
> As mentioned in the cover letter for v2, one major departure from the
> previous snapshot branches referenced in the base driver series is that the
> same MPAM setting are used for kernel-space and user-space. There are pros
> and cons of choosing this policy but I think it is the best thing to start
> with as there are AMD plans for adding a resctrl feature to allow a
> different closid/rmid configuration for user-space from kernel space. The
> AMD feature is called PLZA and is mentioned in this lpc slide deck[1]. This
> gives us a path forward to add support for having the EL1 and EL0 MPAM
> partid/pmg configuration differ from each other.
>
> From James' cover letter:
>
> This is the missing piece to make MPAM usable resctrl in user-space. This has
> shed its debugfs code and the read/write 'event configuration' for the monitors
> to make the series smaller.
>
> This adds the arch code and KVM support first. I anticipate the whole thing
> going via arm64, but if goes via tip instead, the an immutable branch with those
> patches should be easy to do.
>
> Generally the resctrl glue code works by picking what MPAM features it can expose
> from the MPAM drive, then configuring the structs that back the resctrl helpers.
> If your platform is sufficiently Xeon shaped, you should be able to get L2/L3 CPOR
> bitmaps exposed via resctrl. CSU counters work if they are on/after the L3. MBWU
> counters are considerably more hairy, and depend on hueristics around the topology,
> and a bunch of stuff trying to emulate ABMC.
> If it didn't pick what you wanted it to, please share the debug messages produced
> when enabling dynamic debug and booting with:
> | dyndbg="file mpam_resctrl.c +pl"
>
> I've not found a platform that can test all the behaviours around the monitors,
> so this is where I'd expect the most bugs.
>
> The MPAM spec that describes all the system and MMIO registers can be found here:
> https://developer.arm.com/documentation/ddi0598/db/?lang=en
> (Ignored the 'RETIRED' warning - that is just arm moving the documentation around.
>  This document has the best overview)
>
>
> Based on v6.19-rc5
> This series can be retrieved from:
> https://gitlab.arm.com/linux-arm/linux-bh.git mpam_resctrl_glue_v3
>
> v2 can be found at:
> https://lore.kernel.org/linux-arm-kernel/20251219181147.3404071-1-ben.horgan@arm.com/
>
> rfc can be found at:
> https://lore.kernel.org/linux-arm-kernel/20251205215901.17772-1-james.morse@arm.com/
>
> [1] https://lpc.events/event/19/contributions/2093/attachments/1958/4172/resctrl%20Microconference%20LPC%202025%20Tokyo.pdf
>
> Ben Horgan (10):
>   arm_mpam: Use non-atomic bitops when modifying feature bitmap
>   arm64/sysreg: Add MPAMSM_EL1 register
>   KVM: arm64: Preserve host MPAM configuration when changing traps
>   KVM: arm64: Make MPAMSM_EL1 accesses UNDEF
>   arm64: mpam: Initialise and context switch the MPAMSM_EL1 register
>   KVM: arm64: Use kernel-space partid configuration for hypercalls
>   arm_mpam: resctrl: Add rmid index helpers
>   arm_mpam: resctrl: Add kunit test for rmid idx conversions
>   arm_mpam: resctrl: Wait for cacheinfo to be ready
>   arm_mpam: resctrl: Add kunit test for mbw min control generation
>
> Dave Martin (2):
>   arm_mpam: resctrl: Convert to/from MPAMs fixed-point formats
>   arm_mpam: resctrl: Add kunit test for control format conversions
>
> James Morse (30):
>   arm64: mpam: Context switch the MPAM registers
>   arm64: mpam: Re-initialise MPAM regs when CPU comes online
>   arm64: mpam: Advertise the CPUs MPAM limits to the driver
>   arm64: mpam: Add cpu_pm notifier to restore MPAM sysregs
>   arm64: mpam: Add helpers to change a task or cpu's MPAM PARTID/PMG
>     values
>   KVM: arm64: Force guest EL1 to use user-space's partid configuration
>   arm_mpam: resctrl: Add boilerplate cpuhp and domain allocation
>   arm_mpam: resctrl: Sort the order of the domain lists
>   arm_mpam: resctrl: Pick the caches we will use as resctrl resources
>   arm_mpam: resctrl: Implement resctrl_arch_reset_all_ctrls()
>   arm_mpam: resctrl: Add resctrl_arch_get_config()
>   arm_mpam: resctrl: Implement helpers to update configuration
>   arm_mpam: resctrl: Add plumbing against arm64 task and cpu hooks
>   arm_mpam: resctrl: Add CDP emulation
>   arm_mpam: resctrl: Add support for 'MB' resource
>   arm_mpam: resctrl: Add support for csu counters
>   arm_mpam: resctrl: Pick classes for use as mbm counters
>   arm_mpam: resctrl: Pre-allocate free running monitors
>   arm_mpam: resctrl: Pre-allocate assignable monitors
>   arm_mpam: resctrl: Add kunit test for ABMC/CDP interactions
>   arm_mpam: resctrl: Add resctrl_arch_config_cntr() for ABMC use
>   arm_mpam: resctrl: Allow resctrl to allocate monitors
>   arm_mpam: resctrl: Add resctrl_arch_rmid_read() and
>     resctrl_arch_reset_rmid()
>   arm_mpam: resctrl: Add resctrl_arch_cntr_read() &
>     resctrl_arch_reset_cntr()
>   arm_mpam: resctrl: Update the rmid reallocation limit
>   arm_mpam: resctrl: Add empty definitions for assorted resctrl
>     functions
>   arm64: mpam: Select ARCH_HAS_CPU_RESCTRL
>   arm_mpam: resctrl: Call resctrl_init() on platforms that can support
>     resctrl
>   arm_mpam: Generate a configuration for min controls
>   arm_mpam: Quirk CMN-650's CSU NRDY behaviour
>
> Jiapeng Chong (1):
>   arm_mpam: Remove duplicate linux/srcu.h header
>
> Shanker Donthineni (4):
>   arm_mpam: Add quirk framework
>   arm_mpam: Add workaround for T241-MPAM-1
>   arm_mpam: Add workaround for T241-MPAM-4
>   arm_mpam: Add workaround for T241-MPAM-6
>
>  Documentation/arch/arm64/silicon-errata.rst |    9 +
>  arch/arm64/Kconfig                          |    6 +-
>  arch/arm64/include/asm/el2_setup.h          |    3 +-
>  arch/arm64/include/asm/mpam.h               |   98 +
>  arch/arm64/include/asm/resctrl.h            |    2 +
>  arch/arm64/include/asm/thread_info.h        |    3 +
>  arch/arm64/kernel/Makefile                  |    1 +
>  arch/arm64/kernel/cpufeature.c              |   21 +-
>  arch/arm64/kernel/mpam.c                    |   58 +
>  arch/arm64/kernel/process.c                 |    7 +
>  arch/arm64/kvm/hyp/include/hyp/switch.h     |   12 +-
>  arch/arm64/kvm/hyp/nvhe/hyp-main.c          |    8 +
>  arch/arm64/kvm/hyp/vhe/sysreg-sr.c          |   13 +
>  arch/arm64/kvm/sys_regs.c                   |    2 +
>  arch/arm64/tools/sysreg                     |    8 +
>  drivers/resctrl/Kconfig                     |    9 +-
>  drivers/resctrl/Makefile                    |    1 +
>  drivers/resctrl/mpam_devices.c              |  306 ++-
>  drivers/resctrl/mpam_internal.h             |  131 +-
>  drivers/resctrl/mpam_resctrl.c              | 1930 +++++++++++++++++++
>  drivers/resctrl/test_mpam_devices.c         |   66 +
>  drivers/resctrl/test_mpam_resctrl.c         |  426 ++++
>  include/linux/arm_mpam.h                    |   32 +
>  23 files changed, 3119 insertions(+), 33 deletions(-)
>  create mode 100644 arch/arm64/include/asm/mpam.h
>  create mode 100644 arch/arm64/include/asm/resctrl.h
>  create mode 100644 arch/arm64/kernel/mpam.c
>  create mode 100644 drivers/resctrl/mpam_resctrl.c
>  create mode 100644 drivers/resctrl/test_mpam_resctrl.c

Like before, I applied the patches and successfully booted a kernel on
a baremetal Google Cloud C4A instance. I was able to confirm that the
resources we expect were present and I was able to successfully run
the monitor assignment test cases I used to validate ABMC on AMD
systems. (Though I had to hack the driver to pretend there were less
MBWU monitors so that the counter assignment interfaces would become
available.)

My use cases only cover resctrl in the host, so I didn't try any of
the KVM integration, but I can at least say there wasn't any evidence
that it interfered with resctrl.

Tested-by: Peter Newman <peternewman@...gle.com>

Thanks,
-Peter

Powered by blists - more mailing lists