Message-ID: <60543477-842a-60eb-143c-d740c648fde4@amd.com>
Date: Fri, 2 Aug 2024 15:23:17 -0500
From: "Moger, Babu" <bmoger@....com>
To: Peter Newman <peternewman@...gle.com>, babu.moger@....com
Cc: Reinette Chatre <reinette.chatre@...el.com>, corbet@....net,
 fenghua.yu@...el.com, tglx@...utronix.de, mingo@...hat.com, bp@...en8.de,
 dave.hansen@...ux.intel.com, x86@...nel.org, hpa@...or.com,
 paulmck@...nel.org, rdunlap@...radead.org, tj@...nel.org,
 peterz@...radead.org, yanjiewtw@...il.com, kim.phillips@....com,
 lukas.bulwahn@...il.com, seanjc@...gle.com, jmattson@...gle.com,
 leitao@...ian.org, jpoimboe@...nel.org, rick.p.edgecombe@...el.com,
 kirill.shutemov@...ux.intel.com, jithu.joseph@...el.com,
 kai.huang@...el.com, kan.liang@...ux.intel.com,
 daniel.sneddon@...ux.intel.com, pbonzini@...hat.com, sandipan.das@....com,
 ilpo.jarvinen@...ux.intel.com, maciej.wieczor-retman@...el.com,
 linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org, eranian@...gle.com,
 james.morse@....com
Subject: Re: [PATCH v5 00/20] x86/resctrl : Support AMD Assignable Bandwidth
 Monitoring Counters (ABMC)

Hi  Peter,

On 8/2/2024 2:13 PM, Peter Newman wrote:
> Hi Babu,
> 
> On Fri, Aug 2, 2024 at 11:49 AM Moger, Babu <bmoger@....com> wrote:
>>
>> Hi Peter/Reinette,
>>
>> On 8/2/2024 11:13 AM, Reinette Chatre wrote:
>>> Hi Peter,
>>>
>>> On 8/1/24 3:45 PM, Peter Newman wrote:
>>>> However, If we don't expect to see these semantics in any other
>>>> implementation, these semantics could be implicit in the definition of
>>>> a SW assignable counter.
>>>
>>> It is not clear to me how implementation differences between hardware
>>> and software assignment can be hidden from user space. It is possible
>>> to let user space enable individual events and then silently upgrade it
>>> to all events. I see two options here, either "mbm_control" needs to
>>> explicitly show this "silent upgrade" so that user space knows which
>>> events are actually enabled, or "mbm_control" only shows flags/events
>>> enabled
>>> from user space perspective. In the former scenario, this needs more
>>> user space support since a generic user space cannot be confident which
>>> flags are set after writing to "mbm_control". In the latter scenario,
>>> meaning of "num_mbm_cntrs" becomes unclear since user space is expected
>>> to rely on it to know which events can be enabled and if some are
>>> actually "silently enabled" when user space still thinks it needs to be
>>> enabled the number of available counters becomes vague.
>>>
>>> It is not clear to me how to present hardware and software assignable
>>> counters with a single consistent interface. Actually, what if the
>>> "mbm_mode" is what distinguishes how counters are assigned instead of how
>>> it is backed (hw vs sw)? What if, instead of "mbm_cntr_assignable" and
>>> "mbm_cntr_sw_assignable" MBM modes the terms "mbm_cntr_event_assignable"
>>> and "mbm_cntr_group_assignable" is used? Could that replace a
>>> potential "mbm_assign_events" while also supporting user space in
>>> interactions with "mbm_control"?
>>
>> If I understand correctly, current interface might work for both the sw
>> and hw assignments.
>>
>> In case of SW assignment, you need to manage two counters at context
>> switch time. One for total event and one for local event. Basically, you
>> need to calculate delta for both events. You need to do rmid read for
>> both events and then calculate the delta.
>>
>> If the user assigns only one event you do the calculations only for the
>> event user is interested in. That will save cycles as well. In this case
>> "mbm_control" will report as one one event is assigned.
>>
>> In many cases user will not interested in both the events. Also events
>> are configurable so users can get what they want with just one event.
>>
>> Does that make sense?
> 
> I think you've confused soft-RMID for soft-ABMC. Or more likely I've
> confused you by not using consistent terminology.
> 
> soft-RMIDs are simulated by reading the counters of HW RMIDs
> permanently assigned to each CPU at context switch. We found the
> context switch cost of this approach unacceptable.
> 
> soft-ABMC is permanently associating an RMID with the local and total
> counter-pair that will be automatically associated with it when it is
> first loaded into a PQR_ASSOC MSR in a domain, then using the
> mbm_control interface to choose which group to associate with these
> RMIDs. This does not require any context switching work. This
> technique is specific to the behavior of AMD hardware.

Got it.

I assume you have not posted the patches for this yet right?

thanks

Babu Moger

Powered by blists - more mailing lists