linux-kernel - Re: [PATCH v2 0/2] Reset PCIe devices to address DMA problem on kdump with iommu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Wed, 10 Oct 2012 13:22:19 +0900
From:	Takao Indoh <indou.takao@...fujitsu.com>
To:	ddutile@...hat.com
CC:	linux-pci@...r.kernel.org, x86@...nel.org,
	linux-kernel@...r.kernel.org, martin.wilck@...fujitsu.com,
	kexec@...ts.infradead.org, hbabu@...ibm.com, mingo@...hat.com,
	vgoyal@...hat.com, ishii.hironobu@...fujitsu.com, hpa@...or.com,
	bhelgaas@...gle.com, tglx@...utronix.de
Subject: Re: [PATCH v2 0/2] Reset PCIe devices to address DMA problem on kdump
 with iommu

(2012/10/10 1:05), Don Dutile wrote:
> On 10/09/2012 05:03 AM, Takao Indoh wrote:
>> (2012/10/03 22:23), Don Dutile wrote:
>>> On 10/02/2012 03:49 AM, Takao Indoh wrote:
>>>> These patches reset PCIe devices at boot time to address DMA problem on
>>>> kdump with iommu. When "reset_devices" is specified, a hot reset is
>>>> triggered on each PCIe root port and downstream port to reset its
>>>> downstream endpoint.
>>>>
>>>> Background:
>>>> A kdump problem about DMA has been discussed for a long time. That is,
>>>> when a kernel is switched to the kdump kernel DMA derived from first
>>>> kernel affects second kernel. Recently this problem surfaces when iommu
>>>> is used for PCI passthrough on KVM guest. In the case of the machine I
>>>> use, when intel_iommu=on is specified, DMAR error is detected in kdump
>>>> kernel and PCI SERR is also detected. Finally kdump fails because some
>>>> devices does not work correctly.
>>>>
>>>> The root cause is that ongoing DMA from first kernel causes DMAR fault
>>>> because page table of DMAR is initialized while kdump kernel is booting
>>>> up. Therefore to address this problem DMA needs to be stopped before
>>>> DMAR is initialized at kdump kernel boot time. By these patches, PCIe
>>>> devices are reset by hot reset and its DMA is stopped when reset_devices
>>>> is specified. One problem of this solution is that the monitor blacks
>>>> out when VGA controller is reset. So this patch does not reset the port
>>>> whose child endpoint is VGA device.
>>>>
>>>> v2:
>>>> Reset devices in setup_arch() because reset need to be done before
>>>> interrupt remapping is initialized.
>>>>
>>>> v1:
>>>> https://lkml.org/lkml/2012/8/3/160
>>>>
>>>> Thanks,
>>>> Takao Indoh
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>>>> the body of a message to majordomo@...r.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>> Maybe you've tried the following, and I missed a thread on it,
>>> but instead of a somewhat-large, reset hammer, did any one try
>>> just reading all the endpoint-only device CMD register, flip the MasterEnable
>>> bit off, and write it back? .... that would stop all DMA (should stop
>>> all MSI writes as well since they are just another DMA), and then
>>> restart the system?
>>> May also have to do PCI INT Disable as well... and note, that's a PCI 2.3
>>> optional feature.... so devices using INT signalling vs MSI
>>> is just borked on IOMMU/intr-remapping systems... which I would expect
>>> are few. Then again, if this is foolishly done, then reset legacy PCI
>>> busses as the fallback.
>>
>> Just clearing bus master bit and INTx disable bit in setup_arch() did
>> not solve this problem. I still got DMAR error on devices(for exmaple,
>> igb and megaraid_sas).
>>
>> Clearing bus master in setup_arch() and resetting devices in fixup_final
>> like v1 patch is better, DMAR error was not detected. But on certain
>> machine kdump kernel hung up when resetting devices. It seems to be a
>> problem specific to the platform.
>>
>> And, resetting devices in setup_arch() like v2 patch solves all problems
>> I found so far.
>>
>> Thanks,
>> Takao Indoh
>>
> this summary should be in the patch set, so others know/learn
> what was attempted, what failed, and how you reached this working conclusion.

Ok, I'll post new patch with this information.

Thanks,
Takao Indoh

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/