| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <aXrZK-WigVeqxBTT@acelan-Precision-5480>
Date: Thu, 29 Jan 2026 13:45:51 +0800
From: "Chia-Lin Kao (AceLan)" <acelan.kao@...onical.com>
To: Mika Westerberg <mika.westerberg@...ux.intel.com>
Cc: Andreas Noever <andreas.noever@...il.com>,
Mika Westerberg <westeri@...nel.org>, Yehezkel Bernat <YehezkelShB@...il.com>,
linux-usb@...r.kernel.org, linux-kernel@...r.kernel.org,
Gil Fine <gil.fine@...ux.intel.com>
Subject: Re: [PATCH] thunderbolt: Fix PCIe device enumeration with delayed
rescan
On Tue, Jan 27, 2026 at 11:17:01AM +0100, Mika Westerberg wrote:
> On Tue, Jan 27, 2026 at 09:45:13AM +0100, Mika Westerberg wrote:
> > On Tue, Jan 27, 2026 at 01:04:20PM +0800, Chia-Lin Kao (AceLan) wrote:
> > > On Mon, Jan 26, 2026 at 12:56:54PM +0100, Mika Westerberg wrote:
> > > > On Mon, Jan 26, 2026 at 03:48:48PM +0800, Chia-Lin Kao (AceLan) wrote:
> > > > > On Mon, Jan 26, 2026 at 06:42:31AM +0100, Mika Westerberg wrote:
> > > > > > On Mon, Jan 26, 2026 at 11:30:47AM +0800, Chia-Lin Kao (AceLan) wrote:
> > > > > > > Hi,
> > > > > > > On Fri, Jan 23, 2026 at 01:01:12PM +0100, Mika Westerberg wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > On Fri, Jan 23, 2026 at 10:04:11AM +0800, Chia-Lin Kao (AceLan) wrote:
> > > > > > > > > > Can you comment out call to tb_switch_xhci_connect() and see if that
> > > > > > > > > > changes anything?
> > > > > > > > >
> > > > > > > > > Here is what I modified, and the problem becomes a little bit complicated.
> > > > > > > >
> > > > > > > > Okay I see it did not change anything (well this is kind of what I
> > > > > > > > expected). Thanks for trying.
> > > > > > > >
> > > > > > > > I see in your log that the PCIe tunnel is established just fine. It's just
> > > > > > > > that there is no PCIe hotplug happening or it is happening but the PCIe
> > > > > > > > Downstream Port is not waking up.
> > > > > > > >
> > > > > > > > I figured you have following USB4/TB topology, right?
> > > > > > > >
> > > > > > > > AMD Host <-> GR Hub <-> TB3 Hub
> > > > > > > > ^
> > > > > > > > |
> > > > > > > > TB3 Hub
> > > > > > > Should be more like this
> > > > > > > AMD Host <-> Dell TB4 Dock <-> OWC Envoy Express (1-502)
> > > > > > > \
> > > > > > > <-> OWC Envoy Express (1-702)
> > > > > > > or
> > > > > > > AMD Host (1-0, domain1)
> > > > > > > |
> > > > > > > └─ Port 2 ──→ Dell Thunderbolt 4 Dock (1-2)
> > > > > > > ├─ Port 5 ──→ OWC Envoy Express (1-502)
> > > > > > > └─ Port 7 ──→ OWC Envoy Express (1-702)
> > > > > >
> > > > > > Okay so the same ;-)
> > > > > >
> > > > > > > > What if you run 'lspci' after the issue reproduces? Does that bring the
> > > > > > > > missing PCIe devices? I suspect that this is due to older TB3 devices that
> > > > > > > > they may need bit more time to get the PCIe link (going over the tunnel) up
> > > > > > > > and running.
> > > > > > > lspci doesn't bring back the missing tbt storage.
> > > > > >
> > > > > > Forgot to mention that let it (the whole topology) enter runtime suspend
> > > > > > before you run lspci.
> > > > >
> > > > > https://people.canonical.com/~acelan/bugs/tbt_storage/dmesg_lspci.log
> > > > >
> > > > > The behavior is strange, the following 3 devices keep entering D3cold and then comes back
> > > > > to D0 quickly. So, I'm not sure if the lspci do the actions you want.
> > > >
> > > > Yes. I should have mentioned so the lspci is there exactly to trigger
> > > > runtime resume of the topology. I was hoping the PCIe links get
> > > > re-established properly then.
> > > >
> > > > Can you do so that you:
> > > >
> > > > 1. Plug in the dock.
> > > > 2. Plug in the other storage to the dock.
> > > > 3. Block runtime PM from the PCIe Downstream Port that should lead to the
> > > > second storage device PCIe Upstream Port
> > > >
> > > > # echo on > /sys/bus/pci/devices/DEVICE/power/control
> > > >
> > > > 4. Connect the second storage device and enable PCIe tunnel.
> > > >
> > > > Does that make it work each time?
> > > Yes, follow the steps makes it work.
> > >
> > > echo on | sudo tee /sys/bus/pci/devices/*/*/power/control
> > >
> > > Re-plug the dock, need to disable the runpm again.
> >
> > But can you just block it from the PCIe Downstream Port that leads to the
> > "non-working" storage before you enable PCIe tunnel? Not for all the
> > devices.
> >
> > (let me know if you want help locating the correct device).
> >
> > Does it still work?
Here's the full PCI device chain graph:
0000:00:01.2 - AMD PCI Root Port
|
└─ 0000:61:00.0 - Intel Thunderbolt 4 Bridge [Goshen Ridge 2020]
|
└─ 0000:62:02.0 - Intel Thunderbolt 4 Bridge [Goshen Ridge 2020]
|
└─ 0000:83:00.0 - Intel TBT3 Bridge (Upstream Port) [Alpine Ridge LP]
|
└─ 0000:84:01.0 - Intel TBT3 Bridge (Downstream Port) [Alpine Ridge LP]
|
└─ 0000:85:00.0 - Sandisk PC SN740 NVMe SSD (nvme2)
When the tbt storage is not recognized, we don't have 83:00.0 and its
downstream port 84:01.0.
$ ls /sys/bus/pci/devices
0000:00:00.0 0000:00:02.1 0000:00:08.1 0000:00:18.1 0000:00:18.7 0000:62:04.0 0000:c3:00.0 0000:c5:00.5 0000:c7:00.4
0000:00:00.2 0000:00:02.3 0000:00:08.2 0000:00:18.2 0000:61:00.0 0000:a2:00.0 0000:c4:00.0 0000:c5:00.7 0000:c7:00.5
0000:00:01.0 0000:00:02.4 0000:00:08.3 0000:00:18.3 0000:62:00.0 0000:a3:01.0 0000:c5:00.0 0000:c6:00.0 0000:c7:00.6
0000:00:01.1 0000:00:02.5 0000:00:14.0 0000:00:18.4 0000:62:01.0 0000:a4:00.0 0000:c5:00.1 0000:c6:00.1
0000:00:01.2 0000:00:03.0 0000:00:14.3 0000:00:18.5 0000:62:02.0 0000:c1:00.0 0000:c5:00.2 0000:c7:00.0
0000:00:02.0 0000:00:08.0 0000:00:18.0 0000:00:18.6 0000:62:03.0 0000:c2:00.0 0000:c5:00.4 0000:c7:00.3
Disable runpm on 62:02.0, then we have 83:00.0 and its downstream port
84:01.0 and 85:00.0, and then the tbt storage is recognized.
$ echo on | sudo tee /sys/bus/pci/devices/0000:62:02.0/power/control
on
$ ls /sys/bus/pci/devices
0000:00:00.0 0000:00:02.1 0000:00:08.1 0000:00:18.1 0000:00:18.7 0000:62:04.0 0000:a4:00.0 0000:c5:00.1 0000:c6:00.1
0000:00:00.2 0000:00:02.3 0000:00:08.2 0000:00:18.2 0000:61:00.0 0000:83:00.0 0000:c1:00.0 0000:c5:00.2 0000:c7:00.0
0000:00:01.0 0000:00:02.4 0000:00:08.3 0000:00:18.3 0000:62:00.0 0000:84:01.0 0000:c2:00.0 0000:c5:00.4 0000:c7:00.3
0000:00:01.1 0000:00:02.5 0000:00:14.0 0000:00:18.4 0000:62:01.0 0000:85:00.0 0000:c3:00.0 0000:c5:00.5 0000:c7:00.4
0000:00:01.2 0000:00:03.0 0000:00:14.3 0000:00:18.5 0000:62:02.0 0000:a2:00.0 0000:c4:00.0 0000:c5:00.7 0000:c7:00.5
0000:00:02.0 0000:00:08.0 0000:00:18.0 0000:00:18.6 0000:62:03.0 0000:a3:01.0 0000:c5:00.0 0000:c6:00.0 0000:c7:00.6
BTW, rescan also workaround the issue
$ echo 1 | sudo tee /sys/bus/pci/devices/0000:62:02.0/rescan
>
> (+Gil)
>
> There is also one patch that fixes the driver to follow more closely the CM
> guide and that's related to the PCIe tunneling and may actually explain the
> issue you see:
>
> https://lore.kernel.org/linux-usb/20260127094953.GF2275908@black.igk.intel.com/
>
> Note it only does that for USB4 routers so you may need to tune that so
> that it skips that check but there is complication because IIRC LTTSM bits
> do not match the USB4 ones. One thing to try is to just check the USB4 PCIe
> adapter side that it is in detect.
I applied this series directly on top of 6.19-rc6, and the issue
persists.
Here is the modification I tried, but it doesn't work.
diff --git a/drivers/thunderbolt/tunnel.c b/drivers/thunderbolt/tunnel.c
index 51fd05b3e341..0672d3e3c24e 100644
--- a/drivers/thunderbolt/tunnel.c
+++ b/drivers/thunderbolt/tunnel.c
@@ -302,14 +302,38 @@ static int tb_pci_pre_activate(struct tb_tunnel *tunnel)
struct tb_port *up = tunnel->dst_port;
int ret;
- if (!tb_switch_is_usb4(down->sw) || !tb_switch_is_usb4(up->sw))
- return 0;
+ /*
+ * Try checking LTSSM state for both USB4 and TBT3 devices.
+ * Check at least the USB4 side if only one side is USB4.
+ */
+ tb_port_dbg(down, "PCIe downstream port: USB4=%d\n", tb_switch_is_usb4(down->sw));
+ tb_port_dbg(up, "PCIe upstream port: USB4=%d\n", tb_switch_is_usb4(up->sw));
- ret = usb4_pci_port_check_ltssm_state(down, TB_PCIE_LTSSM_DETECT);
- if (ret)
- return ret;
+ if (tb_switch_is_usb4(down->sw)) {
+ tb_port_dbg(down, "Checking PCIe downstream LTSSM state\n");
+ ret = usb4_pci_port_check_ltssm_state(down, TB_PCIE_LTSSM_DETECT);
+ if (ret) {
+ tb_port_warn(down, "PCIe adapter not in detect state: %d\n", ret);
+ return ret;
+ }
+ tb_port_dbg(down, "PCIe downstream adapter in detect state\n");
+ } else {
+ tb_port_dbg(down, "Skipping LTSSM check (not USB4)\n");
+ }
+
+ if (tb_switch_is_usb4(up->sw)) {
+ tb_port_dbg(up, "Checking PCIe upstream LTSSM state\n");
+ ret = usb4_pci_port_check_ltssm_state(up, TB_PCIE_LTSSM_DETECT);
+ if (ret) {
+ tb_port_warn(up, "PCIe adapter not in detect state: %d\n", ret);
+ return ret;
+ }
+ tb_port_dbg(up, "PCIe upstream adapter in detect state\n");
+ } else {
+ tb_port_dbg(up, "Skipping LTSSM check (not USB4)\n");
+ }
- return usb4_pci_port_check_ltssm_state(up, TB_PCIE_LTSSM_DETECT);
+ return 0;
}
static int tb_pci_set_ext_encapsulation(struct tb_tunnel *tunnel, bool enable)
$ sudo dmesg | egrep "PCIe|USB4"
Plug the first tbt storage,
[ 460.465644] [1668] thunderbolt 0000:c7:00.6: Port 4: 8086:15c0 (Revision: 1, TB Version: 1, Type: PCIe (0x100102))
[ 460.891208] [3953] thunderbolt 0000:c7:00.6: 2:12: PCIe downstream port: USB4=1
[ 460.891210] [3953] thunderbolt 0000:c7:00.6: 702:4: PCIe upstream port: USB4=0
[ 460.891212] [3953] thunderbolt 0000:c7:00.6: 2:12: Checking PCIe downstream LTSSM state
[ 460.891327] [3953] thunderbolt 0000:c7:00.6: 2:12: PCIe downstream adapter in detect state
[ 460.891328] [3953] thunderbolt 0000:c7:00.6: 702:4: Skipping LTSSM check (not USB4)
[ 460.891329] [3953] thunderbolt 0000:c7:00.6: activating PCIe Down path from 2:12 to 702:4
[ 460.891849] [3953] thunderbolt 0000:c7:00.6: PCIe Down path activation complete
[ 460.891850] [3953] thunderbolt 0000:c7:00.6: activating PCIe Up path from 702:4 to 2:12
[ 460.892375] [3953] thunderbolt 0000:c7:00.6: PCIe Up path activation complete
[ 461.018893] pci 0000:a2:00.0: [8086:15c0] type 01 class 0x060400 PCIe Switch Upstream Port
[ 461.019746] pci 0000:a2:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:01.2 (capable of 8.000 Gb/s with 2.5 GT/s PCIe x4 link)
[ 461.020620] pci 0000:a3:01.0: [8086:15c0] type 01 class 0x060400 PCIe Switch Downstream Port
[ 461.021846] pci 0000:a4:00.0: [144d:a809] type 00 class 0x010802 PCIe Endpoint
[ 461.022555] pci 0000:a4:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:01.2 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
And then the second tbt storage,
[ 472.025559] [1668] thunderbolt 0000:c7:00.6: Port 4: 8086:15c0 (Revision: 1, TB Version: 1, Type: PCIe (0x100102))
[ 472.451726] [3953] thunderbolt 0000:c7:00.6: 2:11: PCIe downstream port: USB4=1
[ 472.451728] [3953] thunderbolt 0000:c7:00.6: 502:4: PCIe upstream port: USB4=0
[ 472.451729] [3953] thunderbolt 0000:c7:00.6: 2:11: Checking PCIe downstream LTSSM state
[ 472.451851] [3953] thunderbolt 0000:c7:00.6: 2:11: PCIe downstream adapter in detect state
[ 472.451852] [3953] thunderbolt 0000:c7:00.6: 502:4: Skipping LTSSM check (not USB4)
[ 472.451853] [3953] thunderbolt 0000:c7:00.6: activating PCIe Down path from 2:11 to 502:4
[ 472.452373] [3953] thunderbolt 0000:c7:00.6: PCIe Down path activation complete
[ 472.452374] [3953] thunderbolt 0000:c7:00.6: activating PCIe Up path from 502:4 to 2:11
[ 472.452893] [3953] thunderbolt 0000:c7:00.6: PCIe Up path activation complete
My issue should be happening after the PCIe tunnel is activated.
>
> Second thing to try is to disable PCIe ASPM L1. We can do that from CM side
> like we do already in tb_switch_pcie_l1_enable(). If possible you can try
> so that you disable it from the BIOS (but don't use the pcie_aspm= command
> lne parameter).
I can't disable PCIe ASPM L1 from the driver, and there is no option in
the BIOS to switch it off.
Powered by blists - more mailing lists