| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20140714172225.GA10497@laptop.dumpdata.com> Date: Mon, 14 Jul 2014 13:22:25 -0400 From: Konrad Rzeszutek Wilk <konrad.wilk@...cle.com> To: Sander Eikelenboom <linux@...elenboom.it> Cc: konrad@...nel.org, xen-devel@...ts.xenproject.org, boris.ostrovsky@...cle.com, david.vrabel@...rix.com, linux-kernel@...r.kernel.org Subject: Re: [Xen-devel] [PATCH v4] PCI back fixes for 3.17. On Mon, Jul 14, 2014 at 06:37:55PM +0200, Sander Eikelenboom wrote: > > Friday, July 11, 2014, 10:08:47 PM, you wrote: > > > Please see this set of patches which are fixes to Xen pciback > > for 3.17. They are also located at: > > > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git devel/pciback-3.17.v4 > > > These patches do not include the PCI bus reset/slot code as we are still > > discussing that. > > > Konrad Rzeszutek Wilk (5): > > xen-pciback: Document the various parameters and attributes in SysFS > > xen/pciback: Don't deadlock when unbinding. > > xen/pciback: Include the domain id if removing the device whilst still in use > > xen/pciback: Print out the domain owning the device. > > xen/pciback: Remove tons of dereferences > > > Documentation/ABI/testing/sysfs-driver-pciback | 25 +++++++++++++++ > > drivers/xen/xen-pciback/passthrough.c | 9 ++++- > > drivers/xen/xen-pciback/pci_stub.c | 41 +++++++++++++------------ > > drivers/xen/xen-pciback/pciback.h | 7 ++-- > > drivers/xen/xen-pciback/vpci.c | 9 ++++- > > drivers/xen/xen-pciback/xenbus.c | 4 +- > > 6 files changed, 67 insertions(+), 28 deletions(-) > > Hi Konrad / David, > > Thanks for the fixes in this series, i just tested this series and noticed (as > somewhat expected :-) ) it's still lacking a fix for the bug that pciback > doesn't properly free / disown a device from a HVM guest. > > This only happens when using "xl pci-detach domain BDF" > AND > when the guest has more than one pci device attached and you remove only ONE of > them. > > The pci-detach doesn't show an error, the device is removed from the guest, but > it seems it is not internally freed in pciback. > > Below is the sequence and outcome (with some added printk's) for two situations: > A) Guest has only one pci-device passed through which is then removed, as you can see > it's freed in pciback as well. > > B) Guest has two pci-devices passed through from which one is then removed. > After that the second is removed as well. > > >From what i recall Konrad thought it could be due to the guest being a HVM and thus the signaling is different from a > PV-Guest (no pci-front confirming the removal via xenbus). But since it does get freed when it is the last device attached to the guest > i don't know if that's completely true. > It's now semi-fixed when doing the 'xl pci-assignable-remove', that seems to force unregistring the device owner, however something still doesn't seem > right see the second "xl pci-detach" removing the last device from (A), it takes about a minute instead of not more than a second. > > Also note that when on calling "xl pci-assignable-remove" for the last (or only) passed through device for a guest, you don't get the warning messages about > the device being 'in-use' > > Completely below an diff of the added printk's. > > -- > > Sander > > > Ad A) > > root@...0:~# xl pci-list router > Vdev Device > 05.0 0000:02:00.0 > 06.0 0000:00:1b.0 > > root@...0:~# xl pci-assignable-list > > root@...0:~# xl pci-detach router 00:1b.0 > > dmesg shows: > [ 434.839156] pciback 0000:00:1b.0: restoring config space at offset 0x10 (was 0x4, writing 0xf7d30004) > [ 434.841745] pciback 0000:00:1b.0: restoring config space at offset 0xc (was 0x0, writing 0x10) > [ 434.844205] pciback 0000:00:1b.0: restoring config space at offset 0x4 (was 0x100000, writing 0x100006) > > xl dmesg shows: > (XEN) [2014-07-14 16:02:27] memory_map:remove: dom1 gfn=f3070 mfn=f7d30 nr=4 > (XEN) [2014-07-14 16:02:27] io.c:322: d1: unbind: m_gsi=22 g_gsi=40 dev=00:00.6 intx=0 > (XEN) [2014-07-14 16:02:27] io.c:390: d1 unmap: m_irq=22 dev=00:00.6 intx=0 > (XEN) [2014-07-14 16:02:27] [VT-D]iommu.c:1579: d1:PCIe: unmap 0000:00:1b.0 > (XEN) [2014-07-14 16:02:27] [VT-D]iommu.c:1440: d0:PCIe: map 0000:00:1b.0 > > root@...0:~# xl pci-assignable-list > 0000:00:1b.0 > > root@...0:~# xl pci-assignable-remove 00:1b.0 > dmesg shows: > [ 609.246406] xen_pciback: ****** removing device 0000:00:1b.0 while still in-use by domain 1! ****** > [ 609.248985] xen_pciback: ****** driver domain may still access this device's i/o resources! > [ 609.251344] xen_pciback: ****** shutdown driver domain before binding device > [ 609.253291] xen_pciback: ****** to other drivers or domains > [ 609.355083] xen: xen_unregister_device_domain_owner > [ 609.356448] xen: xen_unregister_device_domain_owner > [ 609.357789] xen: xen_unregister_device_domain_owner: ENODEV > [ 609.463125] pciback 0000:00:1b.0: restoring config space at offset 0x10 (was 0x4, writing 0xf7d30004) > [ 609.465692] pciback 0000:00:1b.0: restoring config space at offset 0xc (was 0x0, writing 0x10) > [ 609.468137] pciback 0000:00:1b.0: restoring config space at offset 0x4 (was 0x100000, writing 0x100006) > > root@...0:~# xl pci-list router > Vdev Device > 05.0 0000:02:00.0 > > root@...0:~# xl pci-detach router 02:00.0 > dmesg shows: > [ 930.571300] pciback 0000:02:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x104) > [ 930.574140] pciback 0000:02:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf7c00004) > [ 930.576859] pciback 0000:02:00.0: restoring config space at offset 0x4 (was 0x100000, writing 0x100006) > [ 930.614554] xen-pciback pci-1-0: xen_pcibk_xenbus_remove freeing pdev @ 0xffff880058fcb480 > [ 930.615606] xen-pciback pci-1-0: xen_pcibk_disconnect pdev @ 0xffff880058fcb480 > [ 930.719110] xen: xen_unregister_device_domain_owner > > xl dmesg shows: > (XEN) [2014-07-14 16:10:43] [VT-D]iommu.c:1579: d1:PCIe: unmap 0000:02:00.0 > (XEN) [2014-07-14 16:10:43] [VT-D]iommu.c:1440: d0:PCIe: map 0000:02:00.0 > (XEN) [2014-07-14 16:11:46] memory_map:remove: dom1 gfn=f3074 mfn=f7c00 nr=2 > (XEN) [2014-07-14 16:11:46] io.c:322: d1: unbind: m_gsi=18 g_gsi=36 dev=00:00.5 intx=0 > (XEN) [2014-07-14 16:11:46] io.c:390: d1 unmap: m_irq=18 dev=00:00.5 intx=0 > > root@...0:~# xl pci-assignable-list > 0000:02:00.0 > > root@...0:~# xl pci-assignable-remove 02:00.0 > dmesg shows: > [ 1468.561071] xen: xen_unregister_device_domain_owner > [ 1468.562431] xen: xen_unregister_device_domain_owner: ENODEV > [ 1468.667271] pciback 0000:02:00.0: restoring config space at offset 0x3c (was 0x100, writing 0x104) > [ 1468.670119] pciback 0000:02:00.0: restoring config space at offset 0x10 (was 0x4, writing 0xf7c00004) > [ 1468.672857] pciback 0000:02:00.0: restoring config space at offset 0x4 (was 0x100000, writing 0x100006) > > At this point if you do: #xl pci-assignable-list I presume you don't see anything > Ad B) > > root@...0:~# xl pci-list router > Vdev Device > 05.0 0000:00:1b.0 > > root@...0:~# xl pci-assignable-list > 0000:02:00.0 > > root@...0:~# xl pci-detach router 00:1b.0 > dmesg shows: > [ 199.742668] pciback 0000:00:1b.0: restoring config space at offset 0x10 (was 0x4, writing 0xf7d30004) > [ 199.743527] pciback 0000:00:1b.0: restoring config space at offset 0xc (was 0x0, writing 0x10) > [ 199.744321] pciback 0000:00:1b.0: restoring config space at offset 0x4 (was 0x100000, writing 0x100006) > [ 199.757184] xen-pciback pci-1-0: xen_pcibk_xenbus_remove freeing pdev @ 0xffff8800589fce40 > [ 199.758139] xen-pciback pci-1-0: xen_pcibk_disconnect pdev @ 0xffff8800589fce40 > [ 199.862595] xen: xen_unregister_device_domain_owner > > xl dmesg shows: > (XEN) [2014-07-14 16:28:29] memory_map:remove: dom1 gfn=f3070 mfn=f7d30 nr=4 > (XEN) [2014-07-14 16:28:29] io.c:322: d1: unbind: m_gsi=22 g_gsi=36 dev=00:00.5 intx=0 > (XEN) [2014-07-14 16:28:29] io.c:390: d1 unmap: m_irq=22 dev=00:00.5 intx=0 > (XEN) [2014-07-14 16:28:29] [VT-D]iommu.c:1579: d1:PCIe: unmap 0000:00:1b.0 > (XEN) [2014-07-14 16:28:29] [VT-D]iommu.c:1440: d0:PCIe: map 0000:00:1b.0 > > root@...0:~# xl pci-list router > root@...0:~# xl pci-assignable-list > 0000:00:1b.0 > 0000:02:00.0 > > root@...0:~# xl pci-assignable-remove 00:1b.0 > dmesg shows: > [ 318.827415] xen: xen_unregister_device_domain_owner > [ 318.828771] xen: xen_unregister_device_domain_owner: ENODEV > [ 318.930869] pciback 0000:00:1b.0: restoring config space at offset 0x10 (was 0x4, writing 0xf7d30004) > [ 318.933435] pciback 0000:00:1b.0: restoring config space at offset 0xc (was 0x0, writing 0x10) > [ 318.935877] pciback 0000:00:1b.0: restoring config space at offset 0x4 (was 0x100000, writing 0x100006) > > root@...0:~# xl pci-list router > root@...0:~# xl pci-assignable-list > 0000:02:00.0 > > And if you do: # xl pci-detach router 02:00.0 Do you see it being cleared from pciback? And what do you see in /sys/bus/pci/drivers/pciback ? Thanks! -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists