Date:   Fri, 25 May 2018 11:40:54 +0100
From:   Julien Thierry <julien.thierry@....com>
To:     Daniel Thompson <daniel.thompson@...aro.org>
Cc:     linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org,
        joel@...lfernandes.org, marc.zyngier@....com, mark.rutland@....com,
        christoffer.dall@....com, james.morse@....com,
        catalin.marinas@....com, will.deacon@....com
Subject: Re: [PATCH v4 00/26] arm64: provide pseudo NMI with GICv3



On 25/05/18 11:16, Daniel Thompson wrote:
> On Fri, May 25, 2018 at 10:49:06AM +0100, Julien Thierry wrote:
>> This series is a continuation of the work started by Daniel [1]. The goal
>> is to use GICv3 interrupt priorities to simulate an NMI.
>>
>> To achieve this, set two priorities, one for standard interrupts and
>> another, higher priority, for NMIs. Whenever we want to disable interrupts,
>> we mask the standard priority instead so NMIs can still be raised. Some
>> corner cases though still require to actually mask all interrupts
>> effectively disabling the NMI.
>>
>> Currently, only PPIs and SPIs can be set as NMIs. IPIs being currently
>> hardcoded IRQ numbers, there isn't a generic interface to set SGIs as NMI
>> for now. I don't think there is any reason LPIs should be allowed to be set
>> as NMI as they do not have an active state.
>> When an NMI is active on a CPU, no other NMI can be triggered on the CPU.
>>
>> After the big refactoring I get performances similar to the ones I had
>> in v3[2], reposting old results here:
>>
>> - "hackbench 200 process 1000" (average over 20 runs)
>> +-----------+----------+------------+------------------+
>> |           | native   | PMR guest  | v4.17-rc6 guest  |
>> +-----------+----------+------------+------------------+
>> | PMR host  | 40.0336s |   39.3039s |         39.2044s |
>> | v4.17-rc6 | 40.4040s |   39.6011s |         39.1147s |
>> +-----------+----------+------------+------------------+
>>
>> - Kernel build from defconfig:
>> PMR host:  13m45.743s
>> v4.17-rc6: 13m40.400s
>>
>> I'll try to post more detailed benchmarks later if I find notable
>> differences with the previous version.
> 
> Do you have a public git tree anywhere... I *can* apply 26 patches from
> e-mail but I'd rather pull them!
> 

Yes that makes sense, I'll try to get one set up.

I'll let you know once I have done so.

Cheers,

> 
> Daniel.
> 
>>
>>
>> Requirements to use this:
>> - Have GICv3
>> - SCR_EL3.FIQ is set to 1 when linux runs or have single security state
>> - Select Kernel Feature -> Use ICC system registers for IRQ masking
>>
>>
>> * Patches 1 to 4 aim at applying some alternatives early in the boot
>>    process, including the feature for priority masking.
>>
>> * Patches 5 to 7 and 17 lightly refactor bits of GIC driver to make things
>>    nicer for the rest of the series.
>>
>> * Patches 8 to 10 and 16 ensure the logic of daifflags remains valid
>>    after arch_local_irq flags use ICC_PMR_EL1.
>>
>> * Patches 11 to 14 do some required PMR treatement in order for things to
>>    work when the system uses priority masking.
>>
>> * Patches 15, 18, 19, 20 and 21 actually make the changes to use
>>    ICC_PMR_EL1 for priority masking/unmasking when disabling/enabling
>>    interrupts.
>>
>> * Patches 22 to 26 provide support for pseudo-NMI in the GICv3 driver
>>    when priority masking is enabled.
>>
>>
>> Changes since V3[2]:
>> * Big refactoring. As suggested by Marc Z., some of the bigger patches
>>    needed to be split into smaller one.
>>
>> * Try to reduce the amount of #ifdef for the new feature by introducing
>>    an individual cpufeature for priority masking
>>
>> * Do not track which alternatives have been applied (was a bit dodgy
>>    anyway), and use an alternative for VHE cpu_enable callback
>>
>> * Fix a build failure with arm by adding the correct RPR accessors
>>
>> * Added Suggested-by tags for changes from comming or inspired by Daniel's
>>    series. Do let me know if you feel I missed something and am not giving
>>    you due credit.
>>
>> Changes since V2[3]:
>> * Series rebase to v4.17-rc6
>>
>> * Adapt pathces 1 and 2 to the rework of cpufeatures framework
>>
>> * Use the group0 detection scheme in the GICv3 driver to identify
>>    the priority view, and drop the use of a fake interrupt
>>
>> * Add the case for a GIC configured in a single security state
>>
>> * Use local_daif_restore instead of local_irq_enable the first time
>>    we enable interrupts after a bp hardening in the handling of a kernel
>>    entry. Otherwise PRS.I remains set...
>>
>> Changes since V1[4]:
>> * Series rebased to v4.15-rc8.
>>
>> * Check for arm64_early_features in this_cpu_has_cap (spotted by Suzuki).
>>
>> * Fix issue where debug exception were not masked when enabling debug in
>>    mdscr_el1.
>>
>> Changes since RFC[5]:
>> * The series was rebased to v4.15-rc2 which implied some changes mainly
>>    related to the work on exception entries and daif flags by James Morse.
>>
>>    - The first patch in the previous series was dropped because no longer
>>      applicable.
>>
>>    - With the semantics James introduced of "inheriting" daif flags,
>>      handling of PMR on exception entry is simplified as PMR is not altered
>>      by taking an exception and already inherited from previous state.
>>
>>    - James pointed out that taking a PseudoNMI before reading the FAR_EL1
>>      register should not be allowed as per the TRM (D10.2.29):
>>      "FAR_EL1 is made UNKNOWN on an exception return from EL1."
>>      So in this submission PSR.I bit is cleared only after FAR_EL1 is read.
>>
>> * For KVM, only deal with PMR unmasking/restoring in common code, and VHE
>>    specific code makes sure PSR.I bit is set when necessary.
>>
>> * When detecting the GIC priority view (patch 5), wait for an actual
>>    interrupt instead of trying only once.
>>
>>
>> [1] http://www.spinics.net/lists/arm-kernel/msg525077.html
>> [2] https://lkml.org/lkml/2018/5/21/276
>> [3] https://lkml.org/lkml/2018/1/17/335
>> [4] https://www.spinics.net/lists/arm-kernel/msg620763.html
>> [5] https://www.spinics.net/lists/arm-kernel/msg610736.html
>>
>> Cheers,
>>
>> Julien
>>
>> -->
>>
>> Daniel Thompson (1):
>>    arm64: alternative: Apply alternatives early in boot process
>>
>> Julien Thierry (25):
>>    arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature
>>    arm64: cpufeature: Add cpufeature for IRQ priority masking
>>    arm64: cpufeature: Use alternatives for VHE cpu_enable
>>    irqchip/gic: Unify GIC priority definitions
>>    irqchip/gic: Lower priority of GIC interrupts
>>    irqchip/gic-v3: Remove acknowledge loop
>>    arm64: daifflags: Use irqflags functions for daifflags
>>    arm64: Use daifflag_restore after bp_hardening
>>    arm64: Delay daif masking for user return
>>    arm64: Make PMR part of task context
>>    arm64: Unmask PMR before going idle
>>    arm/arm64: gic-v3: Add helper functions to manage IRQ priorities
>>    arm64: kvm: Unmask PMR before entering guest
>>    arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking
>>    arm64: daifflags: Include PMR in daifflags restore operations
>>    irqchip/gic-v3: Factor group0 detection into functions
>>    irqchip/gic-v3: Do not overwrite PMR value
>>    irqchip/gic-v3: Switch to PMR masking after IRQ acknowledge
>>    arm64: Switch to PMR masking when starting CPUs
>>    arm64: Add build option for IRQ masking via priority
>>    arm64: Detect current view of GIC priorities
>>    irqchip/gic: Add functions to access irq priorities
>>    irqchip/gic-v3: Add base support for pseudo-NMI
>>    irqchip/gic-v3: Provide NMI handlers
>>    irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI
>>
>>   Documentation/arm64/booting.txt        |   5 +
>>   arch/arm/include/asm/arch_gicv3.h      |  33 ++++
>>   arch/arm64/Kconfig                     |  15 ++
>>   arch/arm64/include/asm/alternative.h   |   3 +-
>>   arch/arm64/include/asm/arch_gicv3.h    |  32 ++++
>>   arch/arm64/include/asm/assembler.h     |  17 +-
>>   arch/arm64/include/asm/cpucaps.h       |   3 +-
>>   arch/arm64/include/asm/cpufeature.h    |   2 +
>>   arch/arm64/include/asm/daifflags.h     |  32 ++--
>>   arch/arm64/include/asm/efi.h           |   3 +-
>>   arch/arm64/include/asm/irqflags.h      | 100 ++++++++---
>>   arch/arm64/include/asm/kvm_host.h      |  12 ++
>>   arch/arm64/include/asm/processor.h     |   1 +
>>   arch/arm64/include/asm/ptrace.h        |  13 +-
>>   arch/arm64/kernel/alternative.c        |  30 +++-
>>   arch/arm64/kernel/asm-offsets.c        |   1 +
>>   arch/arm64/kernel/cpufeature.c         |  35 +++-
>>   arch/arm64/kernel/entry.S              |  67 ++++++-
>>   arch/arm64/kernel/head.S               |  35 ++++
>>   arch/arm64/kernel/process.c            |   2 +
>>   arch/arm64/kernel/smp.c                |  12 ++
>>   arch/arm64/kvm/hyp/switch.c            |  17 ++
>>   arch/arm64/mm/fault.c                  |   5 +-
>>   arch/arm64/mm/proc.S                   |  18 ++
>>   drivers/irqchip/irq-gic-common.c       |  10 ++
>>   drivers/irqchip/irq-gic-common.h       |   2 +
>>   drivers/irqchip/irq-gic-v3-its.c       |   2 +-
>>   drivers/irqchip/irq-gic-v3.c           | 318 +++++++++++++++++++++++++++------
>>   include/linux/interrupt.h              |   1 +
>>   include/linux/irqchip/arm-gic-common.h |   6 +
>>   include/linux/irqchip/arm-gic.h        |   5 -
>>   31 files changed, 719 insertions(+), 118 deletions(-)
>>
>> --
>> 1.9.1

-- 
Julien Thierry

Powered by blists - more mailing lists