linux-kernel - Re: [PATCH 3/3] x86/quirks: Add parameter to clear MSIs early on boot

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <12d6175b-7f09-872a-61c4-700e905579c7@canonical.com>
Date:   Thu, 18 Oct 2018 17:13:10 -0300
From:   "Guilherme G. Piccoli" <gpiccoli@...onical.com>
To:     Sinan Kaya <okaya@...nel.org>, linux-pci@...r.kernel.org,
        kexec@...ts.infradead.org, x86@...nel.org
Cc:     linux-kernel@...r.kernel.org, bhelgaas@...gle.com,
        dyoung@...hat.com, bhe@...hat.com, vgoyal@...hat.com,
        tglx@...utronix.de, mingo@...hat.com, bp@...en8.de, hpa@...or.com,
        andi@...stfloor.org, lukas@...ner.de, billy.olsen@...onical.com,
        cascardo@...onical.com, ddstreet@...onical.com,
        fabiomirmar@...onical.com, gavin.guo@...onical.com,
        jay.vosburgh@...onical.com, kernel@...ccoli.net, mfo@...onical.com,
        shan.gavin@...ux.alibaba.com
Subject: Re: [PATCH 3/3] x86/quirks: Add parameter to clear MSIs early on boot



On 18/10/2018 17:08, Sinan Kaya wrote:
> On 10/18/2018 2:37 PM, Guilherme G. Piccoli wrote:
>> We observed a kdump failure in x86 that was narrowed down to MSI irq
>> storm coming from a PCI network device. The bug manifests as a lack of
>> progress in the boot process of kdump kernel, and a flood of kernel
>> messages like:
>>
>> [...]
>> [ 342.265294] do_IRQ: 0.155 No irq handler for vector
>> [ 342.266916] do_IRQ: 0.155 No irq handler for vector
>> [ 347.258422] do_IRQ: 14053260 callbacks suppressed
>> [...]
> 
> These kind of issues are usually fixed by fixing the network driver's
> shutdown routine to ensure that MSI interrupts are cleared there.


Sinan, I'm not sure shutdown handlers for drivers are called in panic
kexec (I remember of an old experiment I did, loading a kernel
with "kexec -p" didn't trigger the handlers).

But this case is even worse, because the NICs were in PCI passthrough
mode, using vfio. So, they were completely unaware of what happened
in the host kernel.

Also, this is spec compliant - system reset events should guarantee the
bits are cleared (although kexec is not exactly a system reset, it's
similar)

Cheers,


Guilherme