linux-kernel - Re: [RFC PATCH 2/2] resctrl2: Arch x86 modules for most of the legacy control/monitor functions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CALPaoCic3zYLxwUET5WCMUCwN8dk-B9JJvuOeUQ2o3hsgNZdQg@mail.gmail.com>
Date:   Thu, 6 Jul 2023 12:22:03 +0200
From:   Peter Newman <peternewman@...gle.com>
To:     "Luck, Tony" <tony.luck@...el.com>
Cc:     James Morse <james.morse@....com>,
        "Yu, Fenghua" <fenghua.yu@...el.com>,
        "Chatre, Reinette" <reinette.chatre@...el.com>,
        Drew Fustini <dfustini@...libre.com>,
        Babu Moger <Babu.Moger@....com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        H Peter Anvin <hpa@...or.com>,
        "shameerali.kolothum.thodi@...wei.com" 
        <shameerali.kolothum.thodi@...wei.com>,
        D Scott Phillips OS <scott@...amperecomputing.com>,
        "carl@...amperecomputing.com" <carl@...amperecomputing.com>,
        "lcherian@...vell.com" <lcherian@...vell.com>,
        "bobo.shaobowang@...wei.com" <bobo.shaobowang@...wei.com>,
        "tan.shaopeng@...itsu.com" <tan.shaopeng@...itsu.com>,
        "xingxin.hx@...nanolis.org" <xingxin.hx@...nanolis.org>,
        "baolin.wang@...ux.alibaba.com" <baolin.wang@...ux.alibaba.com>,
        Jamie Iles <quic_jiles@...cinc.com>,
        Xin Hao <xhao@...ux.alibaba.com>,
        "Pitre, Nicolas" <npitre@...libre.com>,
        Kevin Hilman <khilman@...libre.com>,
        "aricciardi@...libre.com" <aricciardi@...libre.com>,
        "x86@...nel.org" <x86@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "patches@...ts.linux.dev" <patches@...ts.linux.dev>,
        Stephane Eranian <eranian@...gle.com>
Subject: Re: [RFC PATCH 2/2] resctrl2: Arch x86 modules for most of the legacy
 control/monitor functions

Hi Tony,

On Wed, Jul 5, 2023 at 6:46 AM Luck, Tony <tony.luck@...el.com> wrote:
> The mbm_poll() code that makes sure that counters don't wrap is
> doing all the expensive wrmsr(QM_EVTSEL);rdmsr(QM_COUNT)
> once per second to give you the data you want.

I was doing that in the soft RMID series I posted earlier because it
simplified things, but then I had some realizations about how much
error +/- 1 second on the sampling point could result in[1]. We
usually measure the bandwidth rate with a 5-second window, so a
reading that's up to one second old would mean a 20% error in the
bandwidth calculation.

> But existing resctrl
> filesystem doesn't let you do a bulk read. I have some ideas on how
> to provide something better. One question: do you really need that
> snapshot to be system-wide? Or can you live with separate L3-scoped
> snapshots that aren't tightly synchronized with each other?

The most demanding use case is responding to a bandwidth saturation
emergency, where the admin wants a snapshot of all jobs' bandwidth
rates to identify which one(s) to kill. Jobs aren't confined to L3
domains, so I expect it's more important to get a tight snapshot for
an RMID-at-a-time. Going domain-at-a-time could create more races
where a high-bandwidth job migrates at the right time and avoids
detection.

One workaround we've been using to cut down on IPIs is providing a
usermode threshold knob for limiting how old a reported event count
can be, but this is more useful for soft RMIDs where reading a single
CPU's counters makes all RMIDs stored counts current, and IPIs
resulting from userspace reads of different groups in the same domain
are all redundant.

We could probably work with second-old event counts if we knew the
exact time they were recorded so we'd know the exact time delta for
the bandwidth count delta, but that seems like it would be awkward in
the current interface.

-Peter

[1] https://lore.kernel.org/lkml/CALPaoCi_N8JHbP4zFD=ijBh5-=4Q0Ec-LrspYNGGnj4G6A6U0g@mail.gmail.com/