[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <20220314194451.58266-1-mjrosato@linux.ibm.com>
Date: Mon, 14 Mar 2022 15:44:19 -0400
From: Matthew Rosato <mjrosato@...ux.ibm.com>
To: linux-s390@...r.kernel.org
Cc: alex.williamson@...hat.com, cohuck@...hat.com,
schnelle@...ux.ibm.com, farman@...ux.ibm.com, pmorel@...ux.ibm.com,
borntraeger@...ux.ibm.com, hca@...ux.ibm.com, gor@...ux.ibm.com,
gerald.schaefer@...ux.ibm.com, agordeev@...ux.ibm.com,
svens@...ux.ibm.com, frankja@...ux.ibm.com, david@...hat.com,
imbrenda@...ux.ibm.com, vneethv@...ux.ibm.com,
oberpar@...ux.ibm.com, freude@...ux.ibm.com, thuth@...hat.com,
pasic@...ux.ibm.com, joro@...tes.org, will@...nel.org,
pbonzini@...hat.com, corbet@....net, jgg@...dia.com,
kvm@...r.kernel.org, linux-kernel@...r.kernel.org,
iommu@...ts.linux-foundation.org, linux-doc@...r.kernel.org
Subject: [PATCH v4 00/32] KVM: s390: enable zPCI for interpretive execution
Note: A few patches in this series are dependent on Baolu's IOMMU domain ops
split, which is currently in the next branch of linux-iommu. This series
applies on top:
https://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git
Enable interpretive execution of zPCI instructions + adapter interruption
forwarding for s390x KVM vfio-pci. This is done by introducing a new IOMMU
domain for s390x (KVM-managed), indicating via vfio that this IOMMU domain
should be used instead of the default, with subsequent management of the
hardware assists being handled via a new KVM ioctl for zPCI management.
By allowing intepretation of zPCI instructions and firmware delivery of
interrupts to guests, we can significantly reduce the frequency of guest
SIE exits for zPCI. We then see additional gains by handling a hot-path
instruction that can still intercept to the hypervisor (RPCIT) directly
in kvm via the new IOMMU domain, whose map operations update the host
DMA table with pinned guest entries over the specified range.
>From the perspective of guest configuration, you passthrough zPCI devices
in the same manner as before, with intepretation support being used by
default if available in kernel+qemu.
Will reply with a link to the associated QEMU series.
Changelog v3->v4:
v3: https://lore.kernel.org/kvm/20220204211536.321475-1-mjrosato@linux.ibm.com/
- Significant overhaul of the userspace API. Remove all vfio device
feature ioctls. Remove CONFIG_VFIO_PCI_ZDEV, this is once again always
built with vfio-pci for s390; IS_ENABLED checks can instead look at
CONFIG_VFIO_PCI. Most earlier patches in the series could maintain
their reviews, but some needed to be removed due to required code
changes.
- Instead use a KVM ioctl for zPCI management. The API is very similar
to the feature ioctls used in the prior series, with an additional step
to create an association between an iommu domain + KVM + zPCI device.
- Introduce a new iommu domain ops type for s390-iommu, to be used when
KVM manages the IOMMU instead of in response to VFIO mapping ioctls
- Add a iommu method for specifying the type of domain to allocate
- Add a new type to vfio_iommu_type1 (KVM-owned) to trigger the allocation
of the KVM-owned IOMMU domain when zPCI interpretation is requested.
In this case, the KVM-owned type is specified on VFIO_SET_IOMMU.
- Wire the RPCIT intercepts into the new IOMMU domain via the kernel
IOMMU API
- Remove a bunch of unnecessary symbol externs, make the associated
functions static
- Now that we keep a list of zPCI associated with a given KVM, we can do
fh lookup on this list vs the list of all zPCI on the host. We only
need to do a host-wide fh lookup during the initial device<->KVM
association.
Matthew Rosato (32):
s390/sclp: detect the zPCI load/store interpretation facility
s390/sclp: detect the AISII facility
s390/sclp: detect the AENI facility
s390/sclp: detect the AISI facility
s390/airq: pass more TPI info to airq handlers
s390/airq: allow for airq structure that uses an input vector
s390/pci: externalize the SIC operation controls and routine
s390/pci: stash associated GISA designation
s390/pci: export some routines related to RPCIT processing
s390/pci: stash dtsm and maxstbl
s390/pci: add helper function to find device by handle
s390/pci: get SHM information from list pci
s390/pci: return status from zpci_refresh_trans
iommu: introduce iommu_domain_alloc_type and the KVM type
vfio: introduce KVM-owned IOMMU type
vfio-pci/zdev: add function handle to clp base capability
KVM: s390: pci: add basic kvm_zdev structure
iommu/s390: add support for IOMMU_DOMAIN_KVM
KVM: s390: pci: do initial setup for AEN interpretation
KVM: s390: pci: enable host forwarding of Adapter Event Notifications
KVM: s390: mechanism to enable guest zPCI Interpretation
KVM: s390: pci: routines for (dis)associating zPCI devices with a KVM
KVM: s390: pci: provide routines for enabling/disabling interpretation
KVM: s390: pci: provide routines for enabling/disabling interrupt
forwarding
KVM: s390: pci: provide routines for enabling/disabling IOAT assist
KVM: s390: pci: handle refresh of PCI translations
KVM: s390: intercept the rpcit instruction
KVM: s390: add KVM_S390_ZPCI_OP to manage guest zPCI devices
vfio-pci/zdev: add DTSM to clp group capability
KVM: s390: introduce CPU feature for zPCI Interpretation
MAINTAINERS: additional files related kvm s390 pci passthrough
MAINTAINERS: update s390 IOMMU entry
Documentation/virt/kvm/api.rst | 60 +++
MAINTAINERS | 4 +-
arch/s390/include/asm/airq.h | 7 +-
arch/s390/include/asm/kvm_host.h | 7 +
arch/s390/include/asm/kvm_pci.h | 40 ++
arch/s390/include/asm/pci.h | 12 +
arch/s390/include/asm/pci_clp.h | 11 +-
arch/s390/include/asm/pci_dma.h | 3 +
arch/s390/include/asm/pci_insn.h | 31 +-
arch/s390/include/asm/sclp.h | 4 +
arch/s390/include/asm/tpi.h | 13 +
arch/s390/include/uapi/asm/kvm.h | 1 +
arch/s390/kvm/Makefile | 1 +
arch/s390/kvm/interrupt.c | 95 +++-
arch/s390/kvm/kvm-s390.c | 90 +++-
arch/s390/kvm/kvm-s390.h | 10 +
arch/s390/kvm/pci.c | 833 +++++++++++++++++++++++++++++++
arch/s390/kvm/pci.h | 63 +++
arch/s390/kvm/priv.c | 46 ++
arch/s390/pci/pci.c | 31 ++
arch/s390/pci/pci_clp.c | 28 +-
arch/s390/pci/pci_dma.c | 7 +-
arch/s390/pci/pci_insn.c | 15 +-
arch/s390/pci/pci_irq.c | 48 +-
drivers/iommu/Kconfig | 8 +
drivers/iommu/Makefile | 1 +
drivers/iommu/iommu.c | 7 +
drivers/iommu/s390-iommu.c | 53 +-
drivers/iommu/s390-iommu.h | 53 ++
drivers/iommu/s390-kvm-iommu.c | 469 +++++++++++++++++
drivers/s390/char/sclp_early.c | 4 +
drivers/s390/cio/airq.c | 12 +-
drivers/s390/cio/qdio_thinint.c | 6 +-
drivers/s390/crypto/ap_bus.c | 9 +-
drivers/s390/virtio/virtio_ccw.c | 6 +-
drivers/vfio/pci/vfio_pci_zdev.c | 17 +-
drivers/vfio/vfio_iommu_type1.c | 12 +-
include/linux/iommu.h | 12 +
include/uapi/linux/kvm.h | 43 ++
include/uapi/linux/vfio.h | 6 +
include/uapi/linux/vfio_zdev.h | 6 +
41 files changed, 2090 insertions(+), 94 deletions(-)
create mode 100644 arch/s390/include/asm/kvm_pci.h
create mode 100644 arch/s390/kvm/pci.c
create mode 100644 arch/s390/kvm/pci.h
create mode 100644 drivers/iommu/s390-iommu.h
create mode 100644 drivers/iommu/s390-kvm-iommu.c
--
2.27.0
Powered by blists - more mailing lists