linux-kernel - Re: [PROBLEM] c5.metal on AWS fails to kexec after "PCI: Explicitly put devices into D0 when initializing"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <a69dfa26-9ca8-4f3d-ab27-c28f16130c16@amd.com>
Date: Thu, 4 Dec 2025 23:31:01 -0600
From: Mario Limonciello <mario.limonciello@....com>
To: Matthew Ruffell <matthew.ruffell@...onical.com>,
 "bhelgaas@...gle.com" <bhelgaas@...gle.com>
Cc: linux-pci@...r.kernel.org, lkml <linux-kernel@...r.kernel.org>,
 Jay Vosburgh <jay.vosburgh@...onical.com>
Subject: Re: [PROBLEM] c5.metal on AWS fails to kexec after "PCI: Explicitly
 put devices into D0 when initializing"



On 12/4/2025 9:10 PM, Matthew Ruffell wrote:
> Sorry accidentally sent the message.
> 
> The nvme was still in state 0 / PCI_D0:
> 
> [  109.801025] mruffell: vendor: 1d0f, device: 61, state: 0
> [  109.819542] nvme 0000:90:00.0: mruffell: Current PCI device.
> 
> /sys/bus/pci/devices$ ll
> lrwxrwxrwx 1 root root 0 Dec  4 23:24 0000:90:00.0@ ->
> ../../../devices/pci0000:7a/0000:7a:02.0/0000:8d:00.0/0000:8e:01.0/0000:90:00.0
> 
> All of these devices are also state 0. Interesting.
> 
>>> I have a relatively ignorant question.  Can you reproduce with kdump and
>>> a crash too?
>>>
>>> I don't actually know if you configure kdump and then crash the kernel
>>> (say magic sys-rq key), does pci_device_shutdown() get called in order
>>> to do the kexec?  Or because the kernel is already in a crash state is
>>> there just a jump into the crash kernel image location?
>>
> 
> I did check this. I triggered a crash with magic sysrq, and
> pci_device_shutdown()
> was never called. It never printed out my debug messages from
> pci_device_shutdown(), instead it just oopsed and booted straight to the crash
> kernel.
> 
> Thanks,
> Matthew

OK so to me we have two options that you proved both work.

1) Call pci_set_master() during startup.
2) Drop pci_clear_master() for the kexec case during shutdown.

I think we need comments from Bjorn here on which direction is safer 
generally speaking.