| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <dadaeeb9-4008-4450-8b61-e147a2af38b2@linux.dev>
Date: Thu, 27 Nov 2025 20:56:06 -0800
From: Zhu Yanjun <yanjun.zhu@...ux.dev>
To: David Matlack <dmatlack@...gle.com>, Alex Williamson <alex@...zbot.org>
Cc: Adithya Jayachandran <ajayachandra@...dia.com>,
Alex Mastro <amastro@...com>, Alistair Popple <apopple@...dia.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Bjorn Helgaas <bhelgaas@...gle.com>, Chris Li <chrisl@...nel.org>,
David Rientjes <rientjes@...gle.com>,
Jacob Pan <jacob.pan@...ux.microsoft.com>, Jason Gunthorpe <jgg@...dia.com>,
Jason Gunthorpe <jgg@...pe.ca>, Josh Hilke <jrhilke@...gle.com>,
Kevin Tian <kevin.tian@...el.com>, kvm@...r.kernel.org,
Leon Romanovsky <leonro@...dia.com>, linux-kernel@...r.kernel.org,
linux-kselftest@...r.kernel.org, linux-pci@...r.kernel.org,
Lukas Wunner <lukas@...ner.de>, Mike Rapoport <rppt@...nel.org>,
Parav Pandit <parav@...dia.com>, Pasha Tatashin <pasha.tatashin@...een.com>,
Philipp Stanner <pstanner@...hat.com>, Pratyush Yadav <pratyush@...nel.org>,
Saeed Mahameed <saeedm@...dia.com>, Samiullah Khawaja <skhawaja@...gle.com>,
Shuah Khan <shuah@...nel.org>, Tomita Moeko <tomitamoeko@...il.com>,
Vipin Sharma <vipinsh@...gle.com>, William Tu <witu@...dia.com>,
Yi Liu <yi.l.liu@...el.com>, Yunxiang Li <Yunxiang.Li@....com>
Subject: Re: [PATCH 00/21] vfio/pci: Base support to preserve a VFIO device
file across Live Update
在 2025/11/26 11:35, David Matlack 写道:
> This series adds the base support to preserve a VFIO device file across
> a Live Update. "Base support" means that this allows userspace to
> safetly preserve a VFIO device file with LIVEUPDATE_SESSION_PRESERVE_FD
> and retrieve a preserved VFIO device file with
> LIVEUPDATE_SESSION_RETRIEVE_FD, but the device itself is not preserved
> in a fully running state across Live Update.
>
> This series unblocks 2 parallel but related streams of work:
>
> - iommufd preservation across Live Update. This work spans iommufd,
> the IOMMU subsystem, and IOMMU drivers [1]
>
> - Preservation of VFIO device state across Live Update (config space,
> BAR addresses, power state, SR-IOV state, etc.). This work spans both
> VFIO and the core PCI subsystem.
>
> While we need all of the above to fully preserve a VFIO device across a
> Live Update without disrupting the workload on the device, this series
> aims to be functional and safe enough to merge as the first incremental
> step toward that goal.
>
> Areas for Discussion
> --------------------
>
> BDF Stability across Live Update
>
> The PCI support for tracking preserved devices across a Live Update to
> prevent auto-probing relies on PCI segment numbers and BDFs remaining
> stable. For now I have disallowed VFs, as the BDFs assigned to VFs can
> vary depending on how the kernel chooses to allocate bus numbers. For
> non-VFs I am wondering if there is any more needed to ensure BDF
> stability across Live Update.
>
> While we would like to support many different systems and
> configurations in due time (including preserving VFs), I'd like to
> keep this first serses constrained to simple use-cases.
>
> FLB Locking
>
> I don't see a way to properly synchronize pci_flb_finish() with
> pci_liveupdate_incoming_is_preserved() since the incoming FLB mutex is
> dropped by liveupdate_flb_get_incoming() when it returns the pointer
> to the object, and taking pci_flb_incoming_lock in pci_flb_finish()
> could result in a deadlock due to reversing the lock ordering.
>
> FLB Retrieving
>
> The first patch of this series includes a fix to prevent an FLB from
> being retrieved again it is finished. I am wondering if this is the
> right approach or if subsystems are expected to stop calling
> liveupdate_flb_get_incoming() after an FLB is finished.
>
> Testing
> -------
>
> The patches at the end of this series provide comprehensive selftests
> for the new code added by this series. The selftests have been validated
> in both a VM environment using a virtio-net PCIe device, and in a
> baremetal environment on an Intel EMR server with an Intel DSA device.
>
> Here is an example of how to run the new selftests:
Hi, David
ERROR: modpost: "liveupdate_register_file_handler"
[drivers/vfio/pci/vfio-pci-core.ko] undefined!
ERROR: modpost: "vfio_pci_ops" [drivers/vfio/pci/vfio-pci-core.ko]
undefined!
ERROR: modpost: "liveupdate_enabled" [drivers/vfio/pci/vfio-pci-core.ko]
undefined!
ERROR: modpost: "liveupdate_unregister_file_handler"
[drivers/vfio/pci/vfio-pci-core.ko] undefined!
ERROR: modpost: "vfio_device_fops" [drivers/vfio/pci/vfio-pci-core.ko]
undefined!
ERROR: modpost: "vfio_pci_is_intel_display"
[drivers/vfio/pci/vfio-pci-core.ko] undefined!
ERROR: modpost: "vfio_pci_liveupdate_init"
[drivers/vfio/pci/vfio-pci.ko] undefined!
ERROR: modpost: "vfio_pci_liveupdate_cleanup"
[drivers/vfio/pci/vfio-pci.ko] undefined!
make[4]: *** [scripts/Makefile.modpost:147: Module.symvers] Error 1
make[3]: *** [Makefile:1960: modpost] Error 2
After I git clone the source code from the link
https://github.com/dmatlack/linux/tree/liveupdate/vfio/cdev/v1,
I found the above errors when I built the source code.
Perhaps the above errors can be solved by EXPORT_SYMBOL.
But I am not sure if a better solution can solve the above problems or not.
Thanks,
Yanjun.Zhu
>
> vfio_pci_liveupdate_uapi_test:
>
> $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
> $ tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test 0000:00:04.0
> $ tools/testing/selftests/vfio/scripts/cleanup.sh
>
> vfio_pci_liveupdate_kexec_test:
>
> $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
> $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 0000:00:04.0
> $ kexec [...] # NOTE: distro-dependent
>
> $ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
> $ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 0000:00:04.0
> $ tools/testing/selftests/vfio/scripts/cleanup.sh
>
> Dependencies
> ------------
>
> This series was constructed on top of several in-flight series and on
> top of mm-nonmm-unstable [2].
>
> +-- This series
> |
> +-- [PATCH v2 00/18] vfio: selftests: Support for multi-device tests
> | https://lore.kernel.org/kvm/20251112192232.442761-1-dmatlack@google.com/
> |
> +-- [PATCH v3 0/4] vfio: selftests: update DMA mapping tests to use queried IOVA ranges
> | https://lore.kernel.org/kvm/20251111-iova-ranges-v3-0-7960244642c5@fb.com/
> |
> +-- [PATCH v8 0/2] Live Update: File-Lifecycle-Bound (FLB) State
> | https://lore.kernel.org/linux-mm/20251125225006.3722394-1-pasha.tatashin@soleen.com/
> |
> +-- [PATCH v8 00/18] Live Update Orchestrator
> | https://lore.kernel.org/linux-mm/20251125165850.3389713-1-pasha.tatashin@soleen.com/
> |
>
> To simplify checking out the code, this series can be found on GitHub:
>
> https://github.com/dmatlack/linux/tree/liveupdate/vfio/cdev/v1
>
> Changelog
> ---------
>
> v1:
> - Rebase series on top of LUOv8 and VFIO selftests improvements
> - Drop commits to preserve config space fields across Live Update.
> These changes require changes to the PCI layer. For exmaple,
> preserving rbars could lead to an inconsistent device state until
> device BARs addresses are preserved across Live Update.
> - Drop commits to preserve Bus Master Enable on the device. There's no
> reason to preserve this until iommufd preservation is fully working.
> Furthermore, preserving Bus Master Enable could lead to memory
> corruption when the device if the device is bound to the default
> identity-map domain after Live Update.
> - Drop commits to preserve saved PCI state. This work is not needed
> until we are ready to preserve the device's config space, and
> requires more thought to make the PCI state data layout ABI-friendly.
> - Add support to skip auto-probing devices that are preserved by VFIO
> to avoid them getting bound to a different driver by the next kernel.
> - Restrict device preservation further (no VFs, no intel-graphics).
> - Various refactoring and small edits to improve readability and
> eliminate code duplication.
>
> rfc: https://lore.kernel.org/kvm/20251018000713.677779-1-vipinsh@google.com/
>
> Cc: Saeed Mahameed <saeedm@...dia.com>
> Cc: Adithya Jayachandran <ajayachandra@...dia.com>
> Cc: Jason Gunthorpe <jgg@...dia.com>
> Cc: Parav Pandit <parav@...dia.com>
> Cc: Leon Romanovsky <leonro@...dia.com>
> Cc: William Tu <witu@...dia.com>
> Cc: Jacob Pan <jacob.pan@...ux.microsoft.com>
> Cc: Lukas Wunner <lukas@...ner.de>
> Cc: Pasha Tatashin <pasha.tatashin@...een.com>
> Cc: Mike Rapoport <rppt@...nel.org>
> Cc: Pratyush Yadav <pratyush@...nel.org>
> Cc: Samiullah Khawaja <skhawaja@...gle.com>
> Cc: Chris Li <chrisl@...nel.org>
> Cc: Josh Hilke <jrhilke@...gle.com>
> Cc: David Rientjes <rientjes@...gle.com>
>
> [1] https://lore.kernel.org/linux-iommu/20250928190624.3735830-1-skhawaja@google.com/
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/log/?h=mm-nonmm-unstable
>
> David Matlack (12):
> liveupdate: luo_flb: Prevent retrieve() after finish()
> PCI: Add API to track PCI devices preserved across Live Update
> PCI: Require driver_override for incoming Live Update preserved
> devices
> vfio/pci: Notify PCI subsystem about devices preserved across Live
> Update
> vfio: Enforce preserved devices are retrieved via
> LIVEUPDATE_SESSION_RETRIEVE_FD
> vfio/pci: Store Live Update state in struct vfio_pci_core_device
> vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED
> vfio: selftests: Add vfio_pci_liveupdate_uapi_test
> vfio: selftests: Expose iommu_modes to tests
> vfio: selftests: Expose low-level helper routines for setting up
> struct vfio_pci_device
> vfio: selftests: Verify that opening VFIO device fails during Live
> Update
> vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test
>
> Vipin Sharma (9):
> vfio/pci: Register a file handler with Live Update Orchestrator
> vfio/pci: Preserve vfio-pci device files across Live Update
> vfio/pci: Retrieve preserved device files after Live Update
> vfio/pci: Skip reset of preserved device after Live Update
> selftests/liveupdate: Move luo_test_utils.* into a reusable library
> selftests/liveupdate: Add helpers to preserve/retrieve FDs
> vfio: selftests: Build liveupdate library in VFIO selftests
> vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD
> vfio: selftests: Add vfio_pci_liveupdate_kexec_test
>
> MAINTAINERS | 1 +
> drivers/pci/Makefile | 1 +
> drivers/pci/liveupdate.c | 248 ++++++++++++++++
> drivers/pci/pci-driver.c | 12 +-
> drivers/vfio/device_cdev.c | 25 +-
> drivers/vfio/group.c | 9 +
> drivers/vfio/pci/Makefile | 1 +
> drivers/vfio/pci/vfio_pci.c | 11 +-
> drivers/vfio/pci/vfio_pci_core.c | 23 +-
> drivers/vfio/pci/vfio_pci_liveupdate.c | 278 ++++++++++++++++++
> drivers/vfio/pci/vfio_pci_priv.h | 16 +
> drivers/vfio/vfio.h | 13 -
> drivers/vfio/vfio_main.c | 22 +-
> include/linux/kho/abi/pci.h | 53 ++++
> include/linux/kho/abi/vfio_pci.h | 45 +++
> include/linux/liveupdate.h | 3 +
> include/linux/pci.h | 38 +++
> include/linux/vfio.h | 51 ++++
> include/linux/vfio_pci_core.h | 7 +
> kernel/liveupdate/luo_flb.c | 4 +
> tools/testing/selftests/liveupdate/.gitignore | 1 +
> tools/testing/selftests/liveupdate/Makefile | 14 +-
> .../include/libliveupdate.h} | 11 +-
> .../selftests/liveupdate/lib/libliveupdate.mk | 20 ++
> .../{luo_test_utils.c => lib/liveupdate.c} | 43 ++-
> .../selftests/liveupdate/luo_kexec_simple.c | 2 +-
> .../selftests/liveupdate/luo_multi_session.c | 2 +-
> tools/testing/selftests/vfio/Makefile | 23 +-
> .../vfio/lib/include/libvfio/iommu.h | 2 +
> .../lib/include/libvfio/vfio_pci_device.h | 8 +
> tools/testing/selftests/vfio/lib/iommu.c | 4 +-
> .../selftests/vfio/lib/vfio_pci_device.c | 60 +++-
> .../vfio/vfio_pci_liveupdate_kexec_test.c | 255 ++++++++++++++++
> .../vfio/vfio_pci_liveupdate_uapi_test.c | 93 ++++++
> 34 files changed, 1313 insertions(+), 86 deletions(-)
> create mode 100644 drivers/pci/liveupdate.c
> create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c
> create mode 100644 include/linux/kho/abi/pci.h
> create mode 100644 include/linux/kho/abi/vfio_pci.h
> rename tools/testing/selftests/liveupdate/{luo_test_utils.h => lib/include/libliveupdate.h} (80%)
> create mode 100644 tools/testing/selftests/liveupdate/lib/libliveupdate.mk
> rename tools/testing/selftests/liveupdate/{luo_test_utils.c => lib/liveupdate.c} (89%)
> create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c
> create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c
>
--
Best Regards,
Yanjun.Zhu
Powered by blists - more mailing lists