linux-kernel - Re: [PATCH v1 2/9] x86/resctrl: Hold a spinlock in __rmid

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 12 May 2023 15:23:25 +0200
From:   Peter Newman <peternewman@...gle.com>
To:     Reinette Chatre <reinette.chatre@...el.com>
Cc:     Fenghua Yu <fenghua.yu@...el.com>, Babu Moger <babu.moger@....com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org,
        "H. Peter Anvin" <hpa@...or.com>,
        Stephane Eranian <eranian@...gle.com>,
        James Morse <james.morse@....com>,
        linux-kernel@...r.kernel.org, linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v1 2/9] x86/resctrl: Hold a spinlock in __rmid_read() on AMD

Hi Reinette,

On Thu, May 11, 2023 at 11:36 PM Reinette Chatre
<reinette.chatre@...el.com> wrote:
> On 4/21/2023 7:17 AM, Peter Newman wrote:
> > From: Stephane Eranian <eranian@...gle.com>
> >
> > In AMD PQoS Versions 1.0 and 2.0, IA32_QM_EVTSEL MSR is shared by all
> > processors in a QOS domain.  So there's a chance it can read a different
> > event when two processors are reading the counter concurrently.  Add a
> > spinlock to prevent this race.
>
> This is unclear to me. As I understand it this changelog is written as
> though there is a race that is being fixed. I believe that rdtgroup_mutex
> currently protects against such races. I thus at first thought that
> this is a prep patch for the introduction of the new soft RMID feature,
> but instead this new spinlock is used independent of the soft RMID feature.
>
> I think the spinlock is unnecessary when the soft RMID feature is disabled.

My understanding was that the race would happen a lot more when
simultaneously IPI'ing all CPUs in a domain, but I had apparently
overlooked that all of the counter reads were already serialized.


> > + * @lock:    serializes counter reads when QM_EVTSEL MSR is shared per-domain
> >   *
> >   * Members of this structure are accessed via helpers that provide abstraction.
> >   */
> > @@ -333,6 +334,7 @@ struct rdt_hw_domain {
> >       u32                             *ctrl_val;
> >       struct arch_mbm_state           *arch_mbm_total;
> >       struct arch_mbm_state           *arch_mbm_local;
> > +     raw_spinlock_t                  evtsel_lock;
> >  };
>
> Please note the difference between the member name in the struct ("evtsel_lock")
> and its description ("lock").

Will fix, thanks.


> > -static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)
> > +static int __rmid_read(struct rdt_hw_domain *hw_dom, u32 rmid,
> > +                    enum resctrl_event_id eventid, u64 *val)
> >  {
> > +     unsigned long flags;
> >       u64 msr_val;
> >
> > +     if (static_branch_likely(&rmid_read_locked))
>
> Why static_branch_likely() as opposed to static_branch_unlikely()?

I read the documentation for static branches and I agree that unlikely
would make more sense so that the non-locked case is less impacted.

This instance apparently confused my understanding of static branches
and I will need to re-visit all uses of them in this patch series.

>
> > +             raw_spin_lock_irqsave(&hw_dom->evtsel_lock, flags);
> > +
> >       /*
> >        * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured
> >        * with a valid event code for supported resource type and the bits
> > @@ -161,6 +166,9 @@ static int __rmid_read(u32 rmid, enum resctrl_event_id eventid, u64 *val)
> >       wrmsr(MSR_IA32_QM_EVTSEL, eventid, rmid);
> >       rdmsrl(MSR_IA32_QM_CTR, msr_val);
> >
> > +     if (static_branch_likely(&rmid_read_locked))
> > +             raw_spin_unlock_irqrestore(&hw_dom->evtsel_lock, flags);
> > +
>
> If the first "if (static_branch_likely(&rmid_read_locked))" was taken then the second
> if branch _has_ to be taken. It should not be optional to release a lock if it was taken. I
> think it would be more robust if a single test of the static key decides whether the
> spinlock should be used.

Is the concern that the branch value could change concurrently and
result in a deadlock?

I'm curious as to whether this case is performance critical enough to
justify using a static branch. It's clear that we should be using them
in the context switch path, but I'm confused about other places
they're used when there are also memory flags.

-Peter