linux-kernel - Re: [PATCH] nvme: remove multipath module parameter

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z8jk-D3EjEdyBIU5@kbusch-mbp>
Date: Wed, 5 Mar 2025 16:57:44 -0700
From: Keith Busch <kbusch@...nel.org>
To: Christoph Hellwig <hch@....de>
Cc: Hannes Reinecke <hare@...e.de>, Sagi Grimberg <sagi@...mberg.me>,
	Nilay Shroff <nilay@...ux.ibm.com>,
	John Meneghini <jmeneghi@...hat.com>, bmarzins@...hat.com,
	Bryan Gurney <bgurney@...hat.com>, linux-nvme@...ts.infradead.org,
	linux-kernel@...r.kernel.org, Marco Patalano <mpatalan@...hat.com>,
	axboe@...nel.dk
Subject: Re: [PATCH] nvme: remove multipath module parameter

On Thu, Mar 06, 2025 at 12:51:19AM +0100, Christoph Hellwig wrote:
> On Wed, Mar 05, 2025 at 08:17:59AM -0700, Keith Busch wrote:
> > > > Plus there are some NVMe devices out there which _despite_ being PCIe do 
> > > > report NMIC and CMIC set (I won't name names, if you came across them 
> > > > you'll know)
> > > 
> > > ?????
> > > 
> > > NMIC and CMIC is perfectly normal and expected for multiported PCIe.
> > > WTF are you talking about?
> > 
> > Obviously he's not talking about multiported PCIe.
> 
> Why is that obvious?  

No one here would think a multiported device *wouldn't* report CMIC. The
fact Hannes thinks that's a questionable feature for his device gives
away that it is single ported.

> At least based on the stated works he talks about
> PCIe and not about multi-port.  The only not multiported devices I've
> seen that report NMIC and CMIC are a specific firmware so that the
> customer would get multipath behavior, which is a great workaround for
> instable heavily switched fabrics.  Note that multiported isn't always
> obvious as there are quite a few hacks using lane splitting around that
> a normal host can't really see.

In my experience, it's left enabled because of SRIOV, which many of
these devices end up shipping without supporting in PCI space anyway.

> > And he's right, the
> > behavior of a PCIe hot plug is very different and often undesirable when
> > it's under native multipath.
> 
> If you do actual hotplug and expect the device to go away it's indeed
> not desirable.  If you want the same device to come back after switched
> fabric issues it is so desirable that people hack to devices to get it.
> People talked about adding a queue_if_no_path-like parameter to control
> keeping the multipath node alive a lot, but no one has ever invested
> work into actually implementing it.

Not quite the same thing, but kind of related: I proposed this device
missing debounce thing about a year ago:

https://lore.kernel.org/linux-nvme/Y+1aKcQgbskA2tra@kbusch-mbp.dhcp.thefacebook.com/