Message-ID: <CALPaoCjK=VoYriKCGUrtTs=xKxkU34nnwxHKjjy6QVbmD99yhw@mail.gmail.com>
Date: Tue, 20 Jan 2026 16:28:01 +0100
From: Peter Newman <peternewman@...gle.com>
To: Ben Horgan <ben.horgan@....com>
Cc: Reinette Chatre <reinette.chatre@...el.com>, amitsinght@...vell.com, 
	baisheng.gao@...soc.com, baolin.wang@...ux.alibaba.com, 
	carl@...amperecomputing.com, dave.martin@....com, david@...nel.org, 
	dfustini@...libre.com, fenghuay@...dia.com, gshan@...hat.com, 
	james.morse@....com, jonathan.cameron@...wei.com, kobak@...dia.com, 
	lcherian@...vell.com, linux-arm-kernel@...ts.infradead.org, 
	linux-kernel@...r.kernel.org, punit.agrawal@....qualcomm.com, 
	quic_jiles@...cinc.com, rohit.mathew@....com, scott@...amperecomputing.com, 
	sdonthineni@...dia.com, tan.shaopeng@...itsu.com, xhao@...ux.alibaba.com, 
	catalin.marinas@....com, will@...nel.org, corbet@....net, maz@...nel.org, 
	oupton@...nel.org, joey.gouly@....com, suzuki.poulose@....com, 
	kvmarm@...ts.linux.dev
Subject: Re: [PATCH v3 28/47] arm_mpam: resctrl: Add support for csu counters

Hi Ben,

On Fri, Jan 16, 2026 at 11:29 AM Ben Horgan <ben.horgan@....com> wrote:
>
> Hi Reinette, Peter,
>
> On 1/15/26 18:54, Reinette Chatre wrote:
> > Hi Ben,
> >
> > On 1/15/26 7:43 AM, Ben Horgan wrote:
> >> On 1/13/26 23:14, Reinette Chatre wrote:
> >>> On 1/12/26 8:58 AM, Ben Horgan wrote:
> > ...
> >>>> +
> >>>> +          /*
> >>>> +           * Unfortunately, num_rmid doesn't mean anything for
> >>>> +           * mpam, and its exposed to user-space!
> >>>> +           *
> >>>
> >>> The idea of adding a per MON group "num_mon_groups" file has been floated a couple of
> >>> times now. I have not heard any objections against doing something like this.
> >>> https://lore.kernel.org/all/cbe665c2-fe83-e446-1696-7115c0f9fd76@arm.com/
> >>> https://lore.kernel.org/lkml/46767ca7-1f1b-48e8-8ce6-be4b00d129f9@intel.com/
> >>
> >> Hmm, I see now that 'num_rmid' is documented as an upper bound and so
> >> neither 1 or mpam_pmg_max + 1 agree with the documentation.
> >>
> >> "
> >> "num_rmids":
> >>              The number of RMIDs available. This is the
> >>              upper bound for how many "CTRL_MON" + "MON"
> >>              groups can be created.
> >> "
> >
> > Please note that this documentation has been refactored (without changing its
> > meaning). The above quoted text is specific to L3 monitoring and with the
> > addition of telemetry monitoring the relevant text now reads:
> >       The upper bound for how many "CTRL_MON" + "MON" can be created
> >       is the smaller of the L3_MON and PERF_PKG_MON "num_rmids" values.
> >
> >>
> >> So, if I understand correctly you're proposing setting
> >> num_rmids = num_pmg * num_partids on arm platforms and that in the
> >> interim this can then be used to calculate the num_pmg by calculating
> >> num_closid/num_rmid but that a per CTRL_MON num_mon_groups should be
> >> added to make this consistent across architectures?
> >
> > Yes for num_rmids = num_pmg * num_partids.
>
> Ok, I don't really see another option.
>
> The motivation for this is that to me
> > this looks like the value that best matches the num_rmids documentation. I understand
> > the RMID vs PMG is difficult so my proposal is certainly not set in stone and I would like to
> > hear motivation for different interpretations. "calculating num_pmg" is not obvious
> > though. I interpret "num_pmg" here as number of monitor groups per control group and on
> > an Arm system this is indeed num_closid/num_rmids (if num_rmids = num_pmg * num_partids)
> > but on x86 it is just num_rmids. Having user space depend on such computation to determine how
> > many monitor groups per control group would thus require that user space knows whether the
> > underlying system is Arm or x86 and would go against goal of having resctrl as a generic interface.
> >
> > The way forward may be to deprecate (somehow) num_rmids and transition to something
> > like "num_mon_groups" but it is currently vague how "num_mon_groups" may look like. That thread
> > (https://lore.kernel.org/lkml/46767ca7-1f1b-48e8-8ce6-be4b00d129f9@intel.com/) fizzled
> > out after raising a few options how it may look.
> >
> > Another proposal was to add a "mon_id_includes_control_id" to use as another "guide" to
> > determine how many monitoring groups can be created but at the time it seemed an intermediary
> > step for user to determine the number of monitor groups that resctrl can also provide.
> > https://lore.kernel.org/lkml/CALPaoChad6=xqz+BQQd=dB915xhj1gusmcrS9ya+T2GyhTQc5Q@mail.gmail.com/
>
> Just thinking about it now but the "mon_id_includes_control_id" option
> seems the best to me as it is a single bit option that along with
> "num_rmids" let's you know which monitor groups you can create and if
> it's sensible to move monitor groups between CTRL MON groups.
>
> The "num_mon_groups" per CTRL MON group would also need to be
> interpreted together with "num_rmid" to know if it is a global or per
> CTRL MON upper bound. This option also uses multiple files to give the
> same bit of information.
>
> >
> > Making this consistent across architectures is the goal since resctrl aims to be
> > a generic interface. Users should not need to do things like infer which system they
> > are running on by looking at output of resctrl files as mentioned.
> >
> > fwiw ...  there seems to be a usage by Google to compare num_rmids to num_closids to determine
> > how to interact with resctrl:
> > https://lore.kernel.org/lkml/CALPaoCgSO7HzK9BjyM8yL50oPyq9kBj64Nkgyo1WEJrWy5uHUg@mail.gmail.com/
>
> Unfortunately, it looks like we're about to break this heuristic :( At
> least, until a way to get this information generically in resctrl is
> decided upon.

We actually ended up going with the "mon_id_includes_control_id" approach.

The property it represents is rather fundamental to what a monitoring
group actually is and is a low-level implementation detail that is
difficult to hide. Google generally needs support for as many
monitoring IDs as jobs it expects to be able to run on a machine, so
the number of monitoring groups will be routinely maxed out (and there
will be some jobs that are forever stuck in the default group because
no RMIDs were free at the time it started[1])

Thanks,
-Peter

[1] https://lore.kernel.org/lkml/CALPaoCjTwySGX9i7uAtCWLKQpmELKP55xDLJhHmUve8ptsfFTw@mail.gmail.com/

Powered by blists - more mailing lists