| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20240918-ari_no_bus_dev-v2-1-83cfa991082f@linux.ibm.com>
Date: Wed, 18 Sep 2024 11:04:22 +0200
From: Niklas Schnelle <schnelle@...ux.ibm.com>
To: Bjorn Helgaas <bhelgaas@...gle.com>,
Gerald Schaefer <gerald.schaefer@...ux.ibm.com>,
Heiko Carstens <hca@...ux.ibm.com>, Vasily Gorbik <gor@...ux.ibm.com>,
Alexander Gordeev <agordeev@...ux.ibm.com>,
Sven Schnelle <svens@...ux.ibm.com>,
Christian Borntraeger <borntraeger@...ux.ibm.com>,
Gerd Bayer <gbayer@...ux.ibm.com>
Cc: linux-s390@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-pci@...r.kernel.org, Niklas Schnelle <schnelle@...ux.ibm.com>
Subject: [PATCH v2] PCI: s390: Handle ARI on bus without associated struct
pci_dev
On s390 PCI busses are virtualized and the downstream ports are
invisible to the OS and struct pci_bus::self is NULL. This associated
struct pci_dev is however relied upon in pci_ari_enabled() to check
whether ARI is enabled for the bus. ARI is therefor always detected as
disabled.
At the same time firmware on s390 always enables and relies upon ARI
thus causing a mismatch. Moreover with per-PCI function pass-through
there may exist busses with no function with devfn 0. For example
a SR-IOV capable device with two PFs may have separate function
dependency link chains for each of the PFs and their child VFs. In this
case the OS may only see the second PF and its child VFs on a bus
without a devfn 0 function. A situation which is also not supported by
the common pci_configure_ari() code.
Dispite simply being a mismatch this causes problems as some PCI devices
present a different SR-IOV topology depending on PCI_SRIOV_CTRL_ARI.
A similar mismatch may occur with SR-IOV when virtfn_add_bus() creates new
busses with no associated struct pci_dev. Here too pci_ari_enabled()
on these busses would return false even if ARI is actually used.
Prevent both mismatches by moving the ari_enabled flag from struct
pci_dev to struct pci_bus making it independent from struct pci_bus::
self. Let the bus inherit the ari_enabled state from its parent bus when
there is no bridge device such that busses added by virtfn_add_bus()
match their parent. For s390 set ari_enabled when the device supports
ARI in the awareness that all PCIe ports on s390 systems are ARI
capable.
Signed-off-by: Niklas Schnelle <schnelle@...ux.ibm.com>
---
Note: In a comment of the v1 thread I discussed an alternative idea for
a generic solution that would include the Jailhouse hypervisor via
hypervisor_isolated_pci_functions(). As Bjorn correctly pointed out this
more generic solution however lacks a way to indicate that ARI is really
enabled in the hardware. So instead for now I propose to stick with this
patch which only enables this unconditionally on s390x and SR-IOV
virtual busses where the ARI is inherited.
---
Changes in v2:
- Rebased on v6.11
- Link to v1: https://lore.kernel.org/r/20240730-ari_no_bus_dev-v1-1-7de17676f9fe@linux.ibm.com
---
arch/s390/pci/pci_bus.c | 12 ++++++++++++
drivers/pci/pci.c | 4 ++--
drivers/pci/probe.c | 1 +
include/linux/pci.h | 4 ++--
4 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/arch/s390/pci/pci_bus.c b/arch/s390/pci/pci_bus.c
index daa5d7450c7d..021319438dad 100644
--- a/arch/s390/pci/pci_bus.c
+++ b/arch/s390/pci/pci_bus.c
@@ -278,6 +278,18 @@ void pcibios_bus_add_device(struct pci_dev *pdev)
{
struct zpci_dev *zdev = to_zpci(pdev);
+ /*
+ * On s390 PCI busses are virtualized and the bridge
+ * devices are invisible to the OS. Furthermore busses
+ * may exist without a devfn 0 function. Thus the normal
+ * ARI detection does not work. At the same time fw/hw
+ * has always enabled ARI when possible. Reflect the actual
+ * state by setting ari_enabled whenever a device on the bus
+ * supports it.
+ */
+ if (pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_ARI))
+ zdev->zbus->bus->ari_enabled = 1;
+
/*
* With pdev->no_vf_scan the common PCI probing code does not
* perform PF/VF linking.
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ffaaca0978cb..f94a96a6cb30 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3537,11 +3537,11 @@ void pci_configure_ari(struct pci_dev *dev)
if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ARI)) {
pcie_capability_set_word(bridge, PCI_EXP_DEVCTL2,
PCI_EXP_DEVCTL2_ARI);
- bridge->ari_enabled = 1;
+ dev->bus->ari_enabled = 1;
} else {
pcie_capability_clear_word(bridge, PCI_EXP_DEVCTL2,
PCI_EXP_DEVCTL2_ARI);
- bridge->ari_enabled = 0;
+ dev->bus->ari_enabled = 0;
}
}
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index b14b9876c030..c318929438c1 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1143,6 +1143,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent,
if (!bridge) {
child->dev.parent = parent->bridge;
+ child->ari_enabled = parent->ari_enabled;
goto add_dev;
}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 4cf89a4b4cbc..3d1f4a392dd6 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -429,7 +429,6 @@ struct pci_dev {
unsigned int irq_reroute_variant:2; /* Needs IRQ rerouting variant */
unsigned int msi_enabled:1;
unsigned int msix_enabled:1;
- unsigned int ari_enabled:1; /* ARI forwarding */
unsigned int ats_enabled:1; /* Address Translation Svc */
unsigned int pasid_enabled:1; /* Process Address Space ID */
unsigned int pri_enabled:1; /* Page Request Interface */
@@ -679,6 +678,7 @@ struct pci_bus {
struct bin_attribute *legacy_mem; /* Legacy mem */
unsigned int is_added:1;
unsigned int unsafe_warn:1; /* warned about RW1C config write */
+ unsigned int ari_enabled:1; /* ARI forwarding enabled */
};
#define to_pci_bus(n) container_of(n, struct pci_bus, dev)
@@ -2637,7 +2637,7 @@ static inline bool pci_is_dev_assigned(struct pci_dev *pdev)
*/
static inline bool pci_ari_enabled(struct pci_bus *bus)
{
- return bus->self && bus->self->ari_enabled;
+ return bus->ari_enabled;
}
/**
---
base-commit: 98f7e32f20d28ec452afb208f9cffc08448a2652
change-id: 20240724-ari_no_bus_dev-52b2a27f3466
Best regards,
--
Niklas Schnelle
Powered by blists - more mailing lists