[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20240725173624.GA849156@bhelgaas>
Date: Thu, 25 Jul 2024 12:36:24 -0500
From: Bjorn Helgaas <helgaas@...nel.org>
To: Keith Busch <keith.busch@...el.com>
Cc: linux-pci@...r.kernel.org, Bjorn Helgaas <bhelgaas@...gle.com>,
Thomas Gleixner <tglx@...utronix.de>,
LKML <linux-kernel@...r.kernel.org>,
Nirmal Patel <nirmal.patel@...ux.intel.com>,
Jim Harris <james.r.harris@...el.com>,
Paul M Stillwell Jr <paul.m.stillwell.jr@...el.com>,
Blazej Kucman <blazej.kucman@...el.com>
Subject: Re: [PATCHv3 2/2] x86/vmd: Add PCI domain specific LED option
[+cc Nirmal, Jim, Paul, Blazej]
On Tue, Sep 13, 2016 at 09:05:40AM -0600, Keith Busch wrote:
> This patch adds a new function to set PCI domain specific options as
> devices are added. The usage included in this patch is for LED indicator
> control in VMD domains, but may be extended in the future as new domain
> specific options are required.
>
> PCIe LED Slot Control in a VMD domain is repurposed to a non-standard
> implementation. As such, all devices in a VMD domain will be flagged so
> pciehp does not attempt to use LED indicators. This user_led flag
> has pciehp provide a different sysfs entry for user exclusive control
> over the domain's slot indicators.
>
> In order to determine if a bus is within a PCI domain, the patch appends
> a bool to the pci_sysdata structure that the VMD driver sets during
> initialization.
This eventually turned into https://git.kernel.org/linus/3161832d58c7
("x86/PCI: VMD: Request userspace control of PCIe hotplug indicators")
More questions about this, prompted by Blazej's recent regression
report:
https://lore.kernel.org/r/20240722141440.7210-1-blazej.kucman@intel.com
I assume this patch was prompted by NVMe devices behind a VMD? And
the non-standard slot indicator usage is specifically related to VMD
Root Ports? Isn't it possible to add non-NVMe devices behind VMD,
e.g., a switch in an external enclosure where pciehp manages a switch
Downstream Port with standard slot indicators?
I'm wondering if pdev->hotplug_user_indicators should be more narrowly
targeted to just VMD Root Ports.
If there's any possibility of a Downstream Port behind VMD with
standard indicators, users are going to be very confused when the
sysfs "attention" file is basically backwards from normal. IIUC
writing 0 to "attention" when hotplug_user_indicators is set writes 0
("reserved") to AIC, when it would otherwise write 11b ("off").
I'm also wondering whether there's a way to do this in the vmd driver
instead of in arch/x86/pci/common.c, but that's a secondary question.
> arch/x86/include/asm/pci.h | 14 ++++++++++++++
> arch/x86/pci/common.c | 7 +++++++
> arch/x86/pci/vmd.c | 1 +
> 3 files changed, 22 insertions(+)
>
> diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
> index 9ab7507..1411dbe 100644
> --- a/arch/x86/include/asm/pci.h
> +++ b/arch/x86/include/asm/pci.h
> @@ -23,6 +23,9 @@ struct pci_sysdata {
> #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
> void *fwnode; /* IRQ domain for MSI assignment */
> #endif
> +#if IS_ENABLED(CONFIG_VMD)
> + bool vmd_domain; /* True if in Intel VMD domain */
> +#endif
> };
>
> extern int pci_routeirq;
> @@ -56,6 +59,17 @@ static inline void *_pci_root_bus_fwnode(struct pci_bus *bus)
> #define pci_root_bus_fwnode _pci_root_bus_fwnode
> #endif
>
> +static inline bool is_vmd(struct pci_bus *bus)
> +{
> +#if IS_ENABLED(CONFIG_VMD)
> + struct pci_sysdata *sd = bus->sysdata;
> +
> + return sd->vmd_domain;
> +#else
> + return false;
> +#endif
> +}
> +
> /* Can be used to override the logic in pci_scan_bus for skipping
> already-configured bus numbers - to be used for buggy BIOSes
> or architectures with incomplete PCI setup by the loader */
> diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
> index 7b6a9d1..ccf696c 100644
> --- a/arch/x86/pci/common.c
> +++ b/arch/x86/pci/common.c
> @@ -677,6 +677,12 @@ static void set_dma_domain_ops(struct pci_dev *pdev)
> static void set_dma_domain_ops(struct pci_dev *pdev) {}
> #endif
>
> +static void set_dev_domain_options(struct pci_dev *pdev)
> +{
> + if (is_vmd(pdev->bus))
> + pdev->user_leds = 1;
> +}
> +
> int pcibios_add_device(struct pci_dev *dev)
> {
> struct setup_data *data;
> @@ -707,6 +713,7 @@ int pcibios_add_device(struct pci_dev *dev)
> iounmap(data);
> }
> set_dma_domain_ops(dev);
> + set_dev_domain_options(dev);
> return 0;
> }
>
> diff --git a/arch/x86/pci/vmd.c b/arch/x86/pci/vmd.c
> index b814ca6..a021b7b 100644
> --- a/arch/x86/pci/vmd.c
> +++ b/arch/x86/pci/vmd.c
> @@ -596,6 +596,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd)
> .parent = res,
> };
>
> + sd->vmd_domain = true;
> sd->domain = vmd_find_free_domain();
> if (sd->domain < 0)
> return sd->domain;
> --
> 2.7.2
Powered by blists - more mailing lists