Message-ID: <86jzbxqrtw.wl-maz@kernel.org>
Date: Wed, 18 Dec 2024 09:10:51 +0000
From: Marc Zyngier <maz@...nel.org>
To: Rob Herring <robh@...nel.org>
Cc: Anshuman Khandual <anshuman.khandual@....com>,
	linux-arm-kernel@...ts.infradead.org,
	linux-kernel@...r.kernel.org,
	Jonathan Corbet <corbet@....net>,
	Oliver Upton <oliver.upton@...ux.dev>,
	James Morse <james.morse@....com>,
	Suzuki K Poulose <suzuki.poulose@....com>,
	Catalin Marinas <catalin.marinas@....com>,
	Will Deacon <will@...nel.org>,
	Mark Brown <broonie@...nel.org>,
	Mark Rutland <mark.rutland@....com>,
	kvmarm@...ts.linux.dev,
	linux-doc@...r.kernel.org
Subject: Re: [PATCH V3 6/7] arm64/boot: Enable EL2 requirements for FEAT_Debugv8p9

On Tue, 17 Dec 2024 17:53:41 +0000,
Rob Herring <robh@...nel.org> wrote:
> 
> On Tue, Dec 17, 2024 at 2:48 AM Anshuman Khandual
> <anshuman.khandual@....com> wrote:
> >
> > On 12/17/24 05:12, Rob Herring wrote:
> > > On Mon, Dec 16, 2024 at 09:38:30AM +0530, Anshuman Khandual wrote:
> > >> Fine grained trap control for MDSELR_EL1 register needs to be configured in
> > >> HDFGRTR2_EL2, and HDFGWTR2_EL2 registers when kernel enters at EL1, but EL2
> > >> is also present.
> > >
> > > Shouldn't this be "when kernel enters at EL2, and EL3 is also present"?
> >
> > AFAICT - HDFGRTR2_EL2 and HDFGWTR2_EL2 registers configure traps into EL2 when
> > accessed from EL1/EL0, provided all required EL3 trap controls are in place as
> > well. These EL2 based trap configs need to be set before the kernel finally
> > enters EL1. Although there is an assumption about EL3 based trap configs being
> > in place, do we need to mention that in commit message as well. Is not updating
> > booting.rst takes care of all EL3 requirements.
> 
> My point is just I read 'kernel enters at EL1' as meaning the kernel
> booted at EL1 and EL2 is not accessible. Maybe should reworded as
> 'before/if the kernel drops to EL1'

The EL2->EL1 downgrade is internal to the kernel. If we get it wrong,
that's our fault, and there isn't anything to document. This document
is about booting the kernel at any NS EL, so only the "external"
requirements matter.

But I don't think we should be prescriptive about the state of these
registers, as long as the potential traps are correctly handled.

For the above, I'd rather have something like:

"The MDSELR_EL1 register must be freely accessible when the kernel is
 entered at EL1 and that higher ELs are present in the system."

On its own, that should be enough. You can subsequently add:

"This includes HDFGRTR2_EL2, HDFGWTR2_EL2 and MDCR_EL2 when EL2 is
 present, as well as MDCR_EL3 when EL3 is present".

but that's paraphrasing the architecture, and is definitely incomplete
(see the MDSELR_EL1 pseudocode for reference).

> 
> > > Though it is really "If EL3 set FGTEn2 and the kernel is entered in
> > > EL2, then FGT2 must be initialized for MDSELR_EL1."
> > >
> > > If it was me, I'd just plagarize what was written for prior FGT
> > > commits for this code. :)
> >
> > There are many commits that changed __init_el2_fgt() with different description
> > formats. Do you have particular one in mind which can be followed here ? :)
> >
> > >
> > >> This adds a new helper __init_el2_fgt2() initializing this
> > >> new FEAT_FGT2 based fine grained registers.
> > >
> > > "This adds" is the same as saying "This patch/commit adds" which is well
> > > documented to avoid. Use imperative "Add a new helper...". Though it is
> > > clear from the diff that is what you are doing...
> >
> > Sure, will fix it.
> >
> > >
> > >
> > >> MDCR_EL2.EBWE needs to be enabled for additional (beyond 16) breakpoint and
> > >> watchpoint exceptions when kernel enters at EL1, but EL2 is also present.
> > >> This updates __init_el2_debug() as required for FEAT_Debugv8p9.
> > >>
> > >> While here, also update booting.rst with MDCR_EL3 and SCR_EL3 requirements.
> > >>
> > >> Cc: Catalin Marinas <catalin.marinas@....com>
> > >> Cc: Will Deacon <will@...nel.org>
> > >> Cc: Jonathan Corbet <corbet@....net>
> > >> Cc: Marc Zyngier <maz@...nel.org>
> > >> Cc: Oliver Upton <oliver.upton@...ux.dev>
> > >> Cc: linux-arm-kernel@...ts.infradead.org
> > >> Cc: linux-doc@...r.kernel.org
> > >> Cc: linux-kernel@...r.kernel.org
> > >> Cc: kvmarm@...ts.linux.dev
> > >> Signed-off-by: Anshuman Khandual <anshuman.khandual@....com>
> > >> ---
> > >> Changes in V3:
> > >>
> > >> - Dropped MDCR_EL3.TDA boot requirement from documentation (separate series)
> > >> - Dropped MDCR_EL2_EBWE definition as MDCR_EL2 is now defined in tools sysreg
> > >>
> > >> https://lore.kernel.org/all/20241211065425.1106683-1-anshuman.khandual@arm.com/
> > >>
> > >>  Documentation/arch/arm64/booting.rst | 18 ++++++++++++++++++
> > >>  arch/arm64/include/asm/el2_setup.h   | 26 ++++++++++++++++++++++++++
> > >>  2 files changed, 44 insertions(+)
> > >>
> > >> diff --git a/Documentation/arch/arm64/booting.rst b/Documentation/arch/arm64/booting.rst
> > >> index 3278fb4bf219..054cfe1cad18 100644
> > >> --- a/Documentation/arch/arm64/booting.rst
> > >> +++ b/Documentation/arch/arm64/booting.rst
> > >> @@ -288,6 +288,12 @@ Before jumping into the kernel, the following conditions must be met:
> > >>
> > >>      - SCR_EL3.FGTEn (bit 27) must be initialised to 0b1.
> > >>
> > >> +  For CPUs with the Fine Grained Traps (FEAT_FGT2) extension present:
> > >> +
> > >> +  - If EL3 is present and the kernel is entered at EL2:
> > >> +
> > >> +    - SCR_EL3.FGTEn2 (bit 59) must be initialised to 0b1.
> > >> +
> > >>    For CPUs with support for HCRX_EL2 (FEAT_HCX) present:
> > >>
> > >>    - If EL3 is present and the kernel is entered at EL2:
> > >> @@ -322,6 +328,18 @@ Before jumping into the kernel, the following conditions must be met:
> > >>      - ZCR_EL2.LEN must be initialised to the same value for all CPUs the
> > >>        kernel will execute on.
> > >>
> > >> +  For CPUs with FEAT_Debugv8p9 extension present:
> > >> +
> > >> +  - If the kernel is entered at EL1 and EL2 is present:
> > >> +
> > >> +    - HDFGRTR2_EL2.nMDSELR_EL1 (bit 5) must be initialized to 0b1
> > >> +    - HDFGWTR2_EL2.nMDSELR_EL1 (bit 5) must be initialized to 0b1

But then you don't say anything about SCR_EL3.FGTEn2. And what if the
hypervisor wants to trap things in order to emulate on lazy switch
things?

The more I look at those, the more I think these requirements are not
saying what we want to express, which is that some registers must be
accessible one way or another.

> > >> +    - MDCR_EL2.EBWE (bit 43) must be initialized to 0b1
> > >> +
> > >> +  - If EL3 is present:
> > >> +
> > >> +    - MDCR_EL3.EBWE (bit 43) must be initialized to 0b1
> > >> +
> > >>    For CPUs with the Scalable Matrix Extension (FEAT_SME):
> > >>
> > >>    - If EL3 is present:
> > >> diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h
> > >> index 4ef52d7245bb..2fbfe27d38b5 100644
> > >> --- a/arch/arm64/include/asm/el2_setup.h
> > >> +++ b/arch/arm64/include/asm/el2_setup.h
> > >> @@ -105,6 +105,13 @@
> > >>                                              // to own it.
> > >>
> > >>  .Lskip_trace_\@:
> > >> +    mrs     x1, id_aa64dfr0_el1
> > >> +    ubfx    x1, x1, #ID_AA64DFR0_EL1_DebugVer_SHIFT, #4
> > >> +    cmp     x1, #ID_AA64DFR0_EL1_DebugVer_V8P9
> > >> +    b.lt    .Lskip_dbg_v8p9_\@
> > >> +
> > >> +    orr     x2, x2, #MDCR_EL2_EBWE
> > >> +.Lskip_dbg_v8p9_\@:
> > >>      msr     mdcr_el2, x2                    // Configure debug traps
> > >>  .endm
> > >>
> > >> @@ -244,6 +251,24 @@
> > >>  .Lskip_gcs_\@:
> > >>  .endm
> > >>
> > >> +.macro __init_el2_fgt2
> > >> +    mrs     x1, id_aa64mmfr0_el1
> > >> +    ubfx    x1, x1, #ID_AA64MMFR0_EL1_FGT_SHIFT, #4
> > >> +    cmp     x1, #ID_AA64MMFR0_EL1_FGT_FGT2
> > >
> > > We already read this field in __init_el2_fgt, shouldn't we leverage that
> > > and move all this there rather than read the feature reg twice.
> >
> > Should not __init_el2_fgt2() remain separate to contain all future FEAT_FGT2
> > related trap enabled/disable checks ? OTOH reading id_aa64mmfr0_el1 register
> > should improve some performance as well.
> 
> That's the tradeoff. I'll defer to others whether a single id register
> read here is preferred.

Are we *really* talking about *performance* here?  For something that
is executed once per CPU boot? Just keep the damn thing readable, if
at all possible...

	M.

-- 
Without deviation from the norm, progress is not possible.

Powered by blists - more mailing lists