Message-ID: <d8e30c4f-04ef-4ed0-9d06-7f735c1c5e90@arm.com>
Date: Mon, 1 Jul 2024 19:17:37 +0100
From: James Morse <james.morse@....com>
To: Reinette Chatre <reinette.chatre@...el.com>, x86@...nel.org,
 linux-kernel@...r.kernel.org
Cc: Fenghua Yu <fenghua.yu@...el.com>, Thomas Gleixner <tglx@...utronix.de>,
 Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
 H Peter Anvin <hpa@...or.com>, Babu Moger <Babu.Moger@....com>,
 shameerali.kolothum.thodi@...wei.com,
 D Scott Phillips OS <scott@...amperecomputing.com>,
 carl@...amperecomputing.com, lcherian@...vell.com,
 bobo.shaobowang@...wei.com, tan.shaopeng@...itsu.com,
 baolin.wang@...ux.alibaba.com, Jamie Iles <quic_jiles@...cinc.com>,
 Xin Hao <xhao@...ux.alibaba.com>, peternewman@...gle.com,
 dfustini@...libre.com, amitsinght@...vell.com,
 David Hildenbrand <david@...hat.com>, Rex Nie <rex.nie@...uarmicro.com>,
 Dave Martin <dave.martin@....com>
Subject: Re: [PATCH v3 03/38] x86/resctrl: Add a schema format enum and use
 this for fflags

Hi Reinette,

On 28/06/2024 17:43, Reinette Chatre wrote:
> On 6/14/24 7:59 AM, James Morse wrote:
>> resctrl has three types of control, these emerge from the way the
>> architecture initialises a number of properties in struct rdt_resource.
>>
>> A group of these properties need to be set the same on all architectures,
>> it would be better to specify the format the schema entry should use, and
>> allow resctrl to generate all the other properties it needs. This avoids
>> architectures having divergant behaviour here.
> 
> divergant -> divergent ?
> 
>>
>> Add a schema format enum, and as a first use, replace the fflags member
>> of struct rdt_resource.
>>
>> The MBA schema has a different format between AMD and Intel systems.
>> The schema_fmt property is changed by __rdt_get_mem_config_amd() to
>> enable the MBPS format.

>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> index e3edc41882dc..b12307d465bc 100644
>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> @@ -2162,6 +2162,19 @@ static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
>>       return ret;
>>   }
>>   +static u32 fflags_from_resource(struct rdt_resource *r)
>> +{
>> +    switch (r->schema_fmt) {
>> +    case RESCTRL_SCHEMA_BITMAP:
>> +        return RFTYPE_RES_CACHE;
>> +    case RESCTRL_SCHEMA_PERCENTAGE:
>> +    case RESCTRL_SCHEMA_MBPS:
>> +        return RFTYPE_RES_MB;
>> +    }
>> +
>> +    return WARN_ON_ONCE(1);
>> +}
>> +
> 
> The fflags returned specifies which files will be associated with the resource
> in the "info" directory. Basing this on a property of the schema does not look
> right to me. I understand that many of the info files relate to, for example,
> information related to the bitmap used by the cache,

Do we agree that some of them are?

One reason for doing this is it decouples the parsing and management of bitmaps from "this
is the L3 cache", which will make it much easier to support bitmaps on some other kind of
resource.

Ultimately I'd like to expose these to user-space, so that user-space can work out how to
configure resources it doesn't recognise. Today '100' could be a percentage, a bitmap, or
a value in MB/s. Today some knowledge of the control type is needed to work this out.


> but that is not the same for
> info files related to the MBA resource (all info files related to MBA resource
> are not about the schema property format).

Hmmm, because the files min_bandwidth and bandwidth_gran both have bandwidth in their name?

I agree 'delay_linear' and 'thread_throttle_mode' are a bit strange.


> I do not think the type of values of a schema should dictate which files
> appear in the info directory.

Longer term I think this will be a problem. We probably only have 3 types of control:
percentage, bitmap and MB/s... but if each resource on each architecture adds files here
the list will quickly grow. User-space won't be able to work out how to configure a
resource type it hadn't seen before.

This may not be the time - but I think eventually resctrl shouldn't have to care about
what resources the architecture is presenting.
For these files, we may need to duplicate 'min_bandwidth' as 'min_percentage'. MBA would
have both, but any new controls using percentage wouldn't expose them.


> Doesn't MPAM support percentage for cache resources
> and bitmaps for memory resources?

It can have fixed-point-fractions and bitmaps for both caches and memory. Unfortunately
everything in MPAM is optional - the driver converts whatever it finds for memory
bandwidth to a percentage as that is what resctrl and user-space expect.
I can't do the same for cache controls as bitmaps implicitly isolate portions, something
that can't be done with the fractional control. So far everyone has built the bitmaps
because its the easiest implementation - but I have had requests to support the cache
fixed-point-fraction. Doing it as a percentage is least invasive to resctrl...


> Can the fflags rather depend on the resource type itself, by using the rid?

Sure.



Thanks,

James

>> @@ -2182,14 +2195,14 @@ static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
>>       /* loop over enabled controls, these are all alloc_capable */
>>       list_for_each_entry(s, &resctrl_schema_all, list) {
>>           r = s->res;
>> -        fflags = r->fflags | RFTYPE_CTRL_INFO;
>> +        fflags =  fflags_from_resource(r) | RFTYPE_CTRL_INFO;
> 
> (please watch for extra spaces)
> 
>>           ret = rdtgroup_mkdir_info_resdir(s, s->name, fflags);
>>           if (ret)
>>               goto out_destroy;
>>       }
>>         for_each_mon_capable_rdt_resource(r) {
>> -        fflags = r->fflags | RFTYPE_MON_INFO;
>> +        fflags =  fflags_from_resource(r) | RFTYPE_MON_INFO;
> 
> (please watch for extra spaces)
> 
> Reinette

Powered by blists - more mailing lists