linux-kernel - Re: [PATCH] x86/hotplug: Silence APIC only after all irq's are migrated

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <563d77c0-ab5b-df9c-8f4d-b16a0d1211a2@linux.intel.com>
Date:   Thu, 20 Aug 2020 15:11:45 +0300
From:   Mathias Nyman <mathias.nyman@...ux.intel.com>
To:     Ashok Raj <ashok.raj@...el.com>, linux-kernel@...r.kernel.org,
        tglx@...utronix.de
Cc:     Sukumar Ghorai <sukumar.ghorai@...el.com>,
        Srikanth Nandamuri <srikanth.nandamuri@...el.com>,
        Evan Green <evgreen@...omium.org>,
        Bjorn Helgaas <bhelgaas@...gle.com>, stable@...r.kernel.org
Subject: Re: [PATCH] x86/hotplug: Silence APIC only after all irq's are
 migrated

On 15.8.2020 0.38, Ashok Raj wrote:
> When offlining CPU's, fixup_irqs() migrates all interrupts away from the
> outgoing CPU to an online CPU. Its always possible the device sent an
> interrupt to the previous CPU destination. Pending interrupt bit in IRR in
> lapic identifies such interrupts. apic_soft_disable() will not capture any
> new interrupts in IRR. This causes interrupts from device to be lost during
> cpu offline. The issue was found when explicitly setting MSI affinity to a
> CPU and immediately offlining it. It was simple to recreate with a USB
> ethernet device and doing I/O to it while the CPU is offlined. Lost
> interrupts happen even when Interrupt Remapping is enabled.
> 
> Current code does apic_soft_disable() before migrating interrupts.
> 
> native_cpu_disable()
> {
> 	...
> 	apic_soft_disable();
> 	cpu_disable_common();
> 	  --> fixup_irqs(); // Too late to capture anything in IRR.
> }
> 
> Just fliping the above call sequence seems to hit the IRR checks
> and the lost interrupt is fixed for both legacy MSI and when
> interrupt remapping is enabled.
> 
> 
> Fixes: 60dcaad5736f ("x86/hotplug: Silence APIC and NMI when CPU is dead")
> Link: https://lore.kernel.org/lkml/875zdarr4h.fsf@nanos.tec.linutronix.de/
> Signed-off-by: Ashok Raj <ashok.raj@...el.com>
> 
> To: linux-kernel@...r.kernel.org
> To: Thomas Gleixner <tglx@...utronix.de>
> Cc: Sukumar Ghorai <sukumar.ghorai@...el.com>
> Cc: Srikanth Nandamuri <srikanth.nandamuri@...el.com>
> Cc: Evan Green <evgreen@...omium.org>
> Cc: Mathias Nyman <mathias.nyman@...ux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@...gle.com>
> Cc: stable@...r.kernel.org
> ---

This fixes the lost xhci interrupt for me.
Before this patch a msi interupt was lost after ~200 cycles of toggling CPUs
offline/online under heavy usb traffic.

With this patch I ran 3x2000 cycles without any issues (Comet lake, patch on top of 5.8)
Tried both with and without CONFIG_IRQ_REMAP. No issues seen.

Tested-by: Mathias Nyman <mathias.nyman@...ux.intel.com>