| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20240301214117.GA408641@bhelgaas>
Date: Fri, 1 Mar 2024 15:41:17 -0600
From: Bjorn Helgaas <helgaas@...nel.org>
To: Jian-Hong Pan <jhp@...lessos.org>
Cc: Johan Hovold <johan@...nel.org>,
David Box <david.e.box@...ux.intel.com>,
Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>,
Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@...ux.intel.com>,
Mika Westerberg <mika.westerberg@...ux.intel.com>,
Damien Le Moal <dlemoal@...nel.org>,
Nirmal Patel <nirmal.patel@...ux.intel.com>,
Jonathan Derrick <jonathan.derrick@...ux.dev>,
linux-pci@...r.kernel.org, linux-kernel@...r.kernel.org,
linux@...lessos.org
Subject: Re: [PATCH v4 3/3] PCI/ASPM: Fix L1.2 parameters when enable link
state
In subject:
PCI/ASPM: Fix L1.2 parameters before enabling L1.2
On Fri, Feb 16, 2024 at 02:26:44PM +0800, Jian-Hong Pan wrote:
> Currently, when enable link's L1.2 features with __pci_enable_link_state(),
> it configs the link directly without ensuring related L1.2 parameters, such
> as T_POWER_ON, Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD have been
> programmed.
>
> This leads VMD enabled systems' L1.2 of the link between VMD remapped PCIe
> Root Port and NVMe gets wrong configs when a caller tries to enabled it.
This is not VMD-specific.
> Here is a failed example on ASUS B1400CEAE with enabled VMD:
>
> 10000:e0:06.0 PCI bridge: Intel Corporation 11th Gen Core Processor PCIe Controller (rev 01) (prog-if 00 [Normal decode])
> ...
> Capabilities: [200 v1] L1 PM Substates
> L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
> PortCommonModeRestoreTime=45us PortTPowerOnTime=50us
> L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> T_CommonMode=45us LTR1.2_Threshold=101376ns
> L1SubCtl2: T_PwrOn=50us
>
> 10000:e1:00.0 Non-Volatile memory controller: Sandisk Corp WD Blue SN550 NVMe SSD (rev 01) (prog-if 02 [NVM Express])
> ...
> Capabilities: [900 v1] L1 PM Substates
> L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+
> PortCommonModeRestoreTime=32us PortTPowerOnTime=10us
> L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1-
> T_CommonMode=0us LTR1.2_Threshold=0ns
> L1SubCtl2: T_PwrOn=10us
>
> According to PCI Express Base Specification Revision 6.0, Section 5.5.4,
> before enable ASPM L1.2 on the PCIe Root Port and the NVMe, they should be
> programmed with the same LTR1.2_Threshold value. However, they have
> different values in this case.
"PCIe r6.0, sec 5.5.4"
s/before enable/before enabling/
> This patch invokes aspm_calc_l12_info() to program the L1.2 parameters
> properly before enable L1.2 bits of L1 PM Substates Control Register in
> __pci_enable_link_state().
s/This patch invokes/Invoke/
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?id=v6.7#n94
> Also, introduces aspm_get_l1ss_cap() shared
> into aspm_l1ss_init() and __pci_enable_link_state() to get the PCIe
> devices' L1SS capability for aspm_calc_l12_info().
"Also" is always a good clue that something should be split to a
separate patch :)
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394
> Signed-off-by: Jian-Hong Pan <jhp@...lessos.org>
> ---
> v2:
> - Prepare the PCIe LTR parameters before enable L1 Substates
>
> v3:
> - Only enable supported features for the L1 Substates part
>
> v4:
> - Focus on fixing L1.2 parameters, instead of re-initializing whole L1SS
>
> drivers/pci/pcie/aspm.c | 35 ++++++++++++++++++++++++++---------
> 1 file changed, 26 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a39d2ee744cb..42a8c4c194c1 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -588,6 +588,18 @@ static void pcie_aspm_check_latency(struct pci_dev *endpoint)
> }
> }
>
> +static u32 aspm_get_l1ss_cap(struct pci_dev *pdev)
> +{
> + u32 l1ss_cap;
> +
> + pci_read_config_dword(pdev, pdev->l1ss + PCI_L1SS_CAP, &l1ss_cap);
> +
> + if (!(l1ss_cap & PCI_L1SS_CAP_L1_PM_SS))
> + l1ss_cap = 0;
> +
> + return l1ss_cap;
> +}
This is nice but should be a separate patch because it just factors
out existing code, and having it in the same patch obscures the real
point of *this* patch.
> /* Calculate L1.2 PM substate timing parameters */
> static void aspm_calc_l12_info(struct pcie_link_state *link,
> u32 parent_l1ss_cap, u32 child_l1ss_cap)
> @@ -698,15 +710,8 @@ static void aspm_l1ss_init(struct pcie_link_state *link)
> return;
>
> /* Setup L1 substate */
> - pci_read_config_dword(parent, parent->l1ss + PCI_L1SS_CAP,
> - &parent_l1ss_cap);
> - pci_read_config_dword(child, child->l1ss + PCI_L1SS_CAP,
> - &child_l1ss_cap);
> -
> - if (!(parent_l1ss_cap & PCI_L1SS_CAP_L1_PM_SS))
> - parent_l1ss_cap = 0;
> - if (!(child_l1ss_cap & PCI_L1SS_CAP_L1_PM_SS))
> - child_l1ss_cap = 0;
> + parent_l1ss_cap = aspm_get_l1ss_cap(parent);
> + child_l1ss_cap = aspm_get_l1ss_cap(child);
>
> /*
> * If we don't have LTR for the entire path from the Root Complex
> @@ -1367,6 +1372,8 @@ EXPORT_SYMBOL(pci_disable_link_state);
> static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked)
> {
> struct pcie_link_state *link = pcie_aspm_get_link(pdev);
> + struct pci_dev *child = link->downstream, *parent = link->pdev;
> + u32 parent_l1ss_cap, child_l1ss_cap;
>
> if (!link)
> return -EINVAL;
> @@ -1398,6 +1405,16 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked)
> link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1;
> if (state & PCIE_LINK_STATE_L1_2_PCIPM)
> link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1;
> + /*
> + * Ensure L1.2 paramters: Common_Mode_Restore_Times, T_POWER_ON and
> + * LTR_L1.2_THRESHOLD are programmed properly before enable bits for
> + * L1.2, per PCIe r6.0, sec 5.5.4.
s/paramters/parameters/
> + if (state & link->aspm_capable & ASPM_STATE_L1_2_MASK) {
> + parent_l1ss_cap = aspm_get_l1ss_cap(parent);
> + child_l1ss_cap = aspm_get_l1ss_cap(child);
> + aspm_calc_l12_info(link, parent_l1ss_cap, child_l1ss_cap);
Why doesn't this happen already via normal enumeration? It looks like
this path should do it even without this patch:
pcie_aspm_init_link_state
pcie_aspm_cap_init
aspm_l1ss_init
> + }
> pcie_config_aspm_link(link, policy_to_aspm_state(link));
>
> link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0;
> --
> 2.43.2
>
Powered by blists - more mailing lists