linux-kernel - Re: [RFD] resctrl: reassigning a running container's CTRL

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <09029c7a-489a-7054-1ab5-01fa879fb42f@intel.com>
Date:   Fri, 21 Oct 2022 13:09:25 -0700
From:   Reinette Chatre <reinette.chatre@...el.com>
To:     James Morse <james.morse@....com>,
        Peter Newman <peternewman@...gle.com>
CC:     Tony Luck <tony.luck@...el.com>,
        "Yu, Fenghua" <fenghua.yu@...el.com>,
        "Eranian, Stephane" <eranian@...gle.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        "Babu Moger" <Babu.Moger@....com>,
        Gaurang Upasani <gupasani@...gle.com>
Subject: Re: [RFD] resctrl: reassigning a running container's CTRL_MON group

Hi James,

On 10/19/2022 6:57 AM, James Morse wrote:
> Hi Peter,
> 
> On 17/10/2022 11:15, Peter Newman wrote:
>> On Wed, Oct 12, 2022 at 6:55 PM James Morse <james.morse@....com> wrote:
>>> You originally asked:
>>> | Any concerns about the CLOSID-reusing behavior?
>>>
>>> I don't think this will work well with MPAM ... I expect it will mess up the bandwidth
>>> counters.
>>>
>>> MPAM's equivalent to RMID is PMG. While on x86 CLOSID and RMID are independent numbers,
>>> this isn't true for PARTID (MPAM's version of CLOSID) and PMG. The PMG bits effectively
>>> extended the PARTID with bits that aren't used to look up the configuration.
>>>
>>> x86's monitors match only on RMID, and there are 'enough' RMID... MPAMs monitors are more
>>> complicated. I've seen details of a system that only has 1 bit of PMG space.
>>>
>>> While MPAM's bandwidth monitors can match just the PMG, there aren't expected to be enough
>>> unique PMG for every control/monitor group to have a unique value. Instead, MPAM's
>>> monitors are expected to be used with both the PARTID and PMG.
>>>
>>> ('bandwidth monitors' is relevant here, MPAM's 'cache storage utilisation' monitors can't
>>> match on just PMG at all - they have to be told the PARTID too)
>>>
>>>
>>> If you're re-using CLOSID like this, I think you'll end up with noisy measurements on MPAM
>>> systems as the caches hold PARTID/PMG values from before the re-use pattern changed, and
>>> the monitors have to match on both.
> 
>> Yes, that sounds like it would be an issue.
>>
>> Following your refactoring changes, hopefully the MPAM driver could
>> offer alternative methods for managing PARTIDs and PMGs depending on the
>> available hardware resources.
> 
> Mmmm, I don't think anything other than one-partid per control group and one-pmg per
> monitor group makes much sense.
> 
> 
>> If there are a lot more PARTIDs than PMGs, then it would fit well with a
>> user who never creates child MON groups. In case the number of MON
>> groups gets ahead of the number of CTRL_MON groups and you've run out of
>> PMGs, perhaps you would just try to allocate another PARTID and program
>> the same partitioning configuration before giving up.
> 
> User-space can choose to do this.
> If the kernel tries to be clever and do this behind user-space's back, it needs to
> allocate two monitors for this secretly-two-control-groups, and always sum the counters
> before reporting them to user-space.

If I understand this scenario correctly, the kernel is already doing this.
As implemented in mon_event_count() the monitor data of a CTRL_MON group is
the sum of the parent CTRL_MON group and all its child MON groups.

> If monitors are a contended resource, then you may be unable to monitor the
> secretly-two-control-groups group once the kernel has done this.

I am not viewing this as "secretly-two-control-groups" - there would still be
only one parent CTRL_MON group that dictates all the allocations. MON groups already
have a CLOSID (PARTID) property but at this time it is always identical to the parent
CTRL_MON group. The difference introduced is that some of the child MON groups
may have a different CLOSID (PARTID) from the parent.

> 
> I don't think the kernel should try to be too clever here.
> 

That is a fair concern but it may be worth exploring as it seems to address
a few ABI concerns and user space seems to be eyeing using a future "num_closid"
info as a check of "RDT/PQoS" vs "MPAM".

Reinette