Message-ID: <Z7TARX-tFY3mnuU7@kbusch-mbp>
Date: Tue, 18 Feb 2025 10:15:49 -0700
From: Keith Busch <kbusch@...nel.org>
To: John Meneghini <jmeneghi@...hat.com>
Cc: hch@....de, sagi@...mberg.me, bmarzins@...hat.com,
	Bryan Gurney <bgurney@...hat.com>, linux-nvme@...ts.infradead.org,
	linux-kernel@...r.kernel.org, Marco Patalano <mpatalan@...hat.com>,
	axboe@...nel.dk
Subject: Re: [PATCH] nvme: remove multipath module parameter

On Tue, Feb 18, 2025 at 11:31:58AM -0500, John Meneghini wrote:
> On 2/18/25 10:06 AM, Keith Busch wrote:
> > On Thu, Feb 13, 2025 at 03:37:28PM -0500, John Meneghini wrote:
> > > Keith, Christoph and Sagi,
> > > 
> > > This patch has been fully tested and analyzed by Red Hat's QA group and no
> > > unexpected side effects or regressions have been found. Both NVMe/FC and NVMe/TCP
> > > have been tested. Our QE engineer has asked me to report this upstream.
> > 
> > What's the harm in leaving the parameter? *I* use it so I can test both
> > ways without needing to compile multiple kernels.
> 
> LOL.  We've been talking about this since 2017. The goal has always been to remove support for DMMP with NVMe.

I understand that disabling nvme native mp it is required for device
mapper, and I get you want to prevent the possibility of anyone using
dm-mp with nvme, but that isn't the only user that wants to see all
namespace paths.
 
> We want to remove this parameter because it is causing confusion with users and customers who keep trying to use
> DMMP with their multipath NVMe devices.
> 
> We want to remove this parameter because:
> 
> 1) the upstream kernel does not support multipath nvme devices without CONFIG_NVME_MULTIPATH enabled

What do you mean by "support"? I assume you mean no one upstream will
help you debug your problems if you've set yourself up that way, and
that's probably true. The kernel currently doesn't stop you from doing
this though, so it's supported in that sense. Some people are fine doing
this on their own, they're not seeking upstream help. Changing this
would break userspace because it makes the driver fail to create device
nodes that used to show up.

> 2) the upstream kernel does not support multipath nvme devices when core.nvme_multipath is set to N
> 3) Non-multipath nvme devies are supported just fine with core.nvme_multipath is set to Y
> 
> You don't need set core.nvme_multipath to N to test your devices both ways w/o recompiling the kernel.
> All of the code paths involved here are controlled by NVME_CTRL_CMIC_MULTI_CTRL and setting core.nvme_multipath
> to N doesn't do anything to help your single path NVMe devices. It doesn't remove multipath.c, reduce the code
> path length or do anything else to optimize your non-NVME_CTRL_CMIC_MULTI_CTRL devices.  All it does is provide
> an escape hatch to disable the incore multipath scheduler start creating multiple /dev/nvme%d/n%d entries so
> that DMMP can be used with multipath capable NVMe devices.
> 
> Personally, I'd like to remove CONFIG_NVME_MULTIPATH as well. It's just another source of confusion. Most users
> are running Linux with the the default settings for NVME_MULTIPATH. This is what Red Hat customers use and that's
> what's used upstream.  So what's the point?

There are devices that report CMIC and NMIC despite being single path,
perhaps as some vestigial sr-iov feature. That adds an unnecessary layer
for all IO to go through. Having the param makes it easy to test both
possible driver paths. In production though, I'd expect to just disable
the CONFIG option if that's the behavior someoone wants, so I think the
config option ought to stay.

Powered by blists - more mailing lists