Message-ID: <92bcab75-72c6-46d4-97a2-119e7124c90c@amd.com>
Date: Tue, 20 May 2025 18:25:32 -0500
From: "Moger, Babu" <bmoger@....com>
To: Reinette Chatre <reinette.chatre@...el.com>, babu.moger@....com,
 Peter Newman <peternewman@...gle.com>
Cc: corbet@....net, tony.luck@...el.com, tglx@...utronix.de,
 mingo@...hat.com, bp@...en8.de, dave.hansen@...ux.intel.com,
 james.morse@....com, dave.martin@....com, fenghuay@...dia.com,
 x86@...nel.org, hpa@...or.com, paulmck@...nel.org,
 akpm@...ux-foundation.org, thuth@...hat.com, rostedt@...dmis.org,
 ardb@...nel.org, gregkh@...uxfoundation.org, daniel.sneddon@...ux.intel.com,
 jpoimboe@...nel.org, alexandre.chartre@...cle.com,
 pawan.kumar.gupta@...ux.intel.com, thomas.lendacky@....com,
 perry.yuan@....com, seanjc@...gle.com, kai.huang@...el.com,
 xiaoyao.li@...el.com, kan.liang@...ux.intel.com, xin3.li@...el.com,
 ebiggers@...gle.com, xin@...or.com, sohil.mehta@...el.com,
 andrew.cooper3@...rix.com, mario.limonciello@....com,
 linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
 maciej.wieczor-retman@...el.com, eranian@...gle.com, Xiaojian.Du@....com,
 gautham.shenoy@....com
Subject: Re: [PATCH v13 00/27] x86/resctrl : Support AMD Assignable Bandwidth
 Monitoring Counters (ABMC)

Hi Reinette,

On 5/20/2025 1:23 PM, Reinette Chatre wrote:
> Hi Babu,
> 
> On 5/20/25 10:51 AM, Moger, Babu wrote:
>> Hi Reinette,
>>
>> On 5/20/25 11:06, Reinette Chatre wrote:
>>> Hi Babu,
>>>
>>> On 5/20/25 8:28 AM, Moger, Babu wrote:
>>>> On 5/19/25 10:59, Peter Newman wrote:
>>>>> On Fri, May 16, 2025 at 12:52 AM Babu Moger <babu.moger@....com> wrote:
>>>
>>> ...
>>>
>>>>>> /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs: Reports the number of monitoring
>>>>>> counters available for assignment.
>>>>>
>>>>> Earlier I discussed with Reinette[1] what num_mbm_cntrs should
>>>>> represent in a "soft-ABMC" implementation where assignment is
>>>>> implemented by assigning an RMID, which would result in all events
>>>>> being assigned at once.
>>>>>
>>>>> My main concern is how many "counters" you can assign by assigning
>>>>> RMIDs. I recall Reinette proposed reporting the number of groups which
>>>>> can be assigned separately from counters which can be assigned.
>>>>
>>>> More context may be needed here. Currently, num_mbm_cntrs indicates the
>>>> number of counters available per domain, which is 32.
>>>>
>>>> At the moment, we can assign 2 counters to each group, meaning each RMID
>>>> can be associated with 2 hardware counters. In theory, it's possible to
>>>> assign all 32 hardware counters to a group—allowing one RMID to be linked
>>>> with up to 32 counters. However, we currently lack the interface to
>>>> support that level of assignment.
>>>>
>>>> For now, the plan is to support basic assignment and expand functionality
>>>> later once we have the necessary data structure and requirements.
>>>
>>> Looks like some requirements did not make it into this implementation.
>>> Do you recall the discussion that resulted in you writing [2]? Looks like
>>> there is a question to Peter in there on how to determine how many "counters"
>>> are available in soft-ABMC. I interpreted [3] at that time to mean that this
>>> information would be available in a future AMD publication.
>>
>> We already have a method to determine the number of counters in soft-ABMC
>> mode, which Peter has addressed [4].
>>
>> [4]
>> https://lore.kernel.org/lkml/20250203132642.2746754-1-peternewman@google.com/
>>
>> This appears to be more of a workaround, and I doubt it will be included
>> in any official AMD documentation. Additionally, the long-term direction
>> is moving towards ABMC.
>>
>> I don’t believe this workaround needs to be part of the current series. It
>> can be added later when soft-ABMC is implemented.
> 
> Agreed. What about the plans described in [2]? (Thanks to Peter for
> catching this!).
> 
> It is important to keep track of requirements while working on a feature to
> ensure that the implementation supports the planned use cases. Re-reading that
> thread it is not clear to me how soft-ABMC's per-group assignment would look.
> Could you please share how you see it progress from this implementation?
> This includes the single event vs. multiple event assignment. I would like to
> highlight that this is not a request for this to be supported in this implementation
> but there needs to be a plan for how this can be supported on top of interfaces
> established by this work.
> 

Here’s my current understanding of soft-ABMC. Peter may have a more 
in-depth perspective on this.

Soft-ABMC:
a. num_mbm_cntrs: This is a software-defined limit based on the number 
of active RMIDs that can be supported. The value can be obtained using 
the code referenced in [4].

b. Assignments: No hardware configuration is required. We simply need to 
ensure that no more than num_mbm_cntrs RMIDs are active at any given time.

c. Configuration: Controlled via /info/L3_MON/mbm_total_bytes_config and 
mbm_local_bytes_config.

d. Events: Only two events can be assigned(local and total).

ABMC:
a. num_mbm_cntrs: This is defined by the hardware.
b. Assignments: Requires special MSR writes to assign counters.
c. Configuration: Comes from /info/L3_MON/counter_configs/.
d. Events: More than two events can be assigned to a group (currently up 
to 2).

Commonalities:
a. Assignments can be either exclusive or shared in both these modes.

Given these, I believe we can easily accommodate soft-ABMC in this 
interface.

>>>
>>> [2] https://lore.kernel.org/lkml/afb99efe-0de2-f7ad-d0b8-f2a0ea998efd@amd.com/
>>> [3] https://lore.kernel.org/lkml/CALPaoCg3KpF94g2MEmfP_Ro2mQZYFA8sKVkmb+7isotKNgdY9A@mail.gmail.com/
>>
> 
> 

Powered by blists - more mailing lists