Message-ID: <20240710142238.00007295@Huawei.com>
Date: Wed, 10 Jul 2024 14:22:38 +0100
From: Jonathan Cameron <Jonathan.Cameron@...wei.com>
To: Dan Williams <dan.j.williams@...el.com>
CC: James Bottomley <James.Bottomley@...senpartnership.com>,
	<ksummit@...ts.linux.dev>, <linux-cxl@...r.kernel.org>,
	<linux-rdma@...r.kernel.org>, <netdev@...r.kernel.org>, <jgg@...dia.com>
Subject: Re: [MAINTAINERS SUMMIT] Device Passthrough Considered Harmful?

On Tue, 9 Jul 2024 15:15:13 -0700
Dan Williams <dan.j.williams@...el.com> wrote:

> James Bottomley wrote:
> > > The upstream discussion has yielded the full spectrum of positions on
> > > device specific functionality, and it is a topic that needs cross-
> > > kernel consensus as hardware increasingly spans cross-subsystem
> > > concerns. Please consider it for a Maintainers Summit discussion.  
> > 
> > I'm with Greg on this ... can you point to some of the contrary
> > positions?  
> 
> This thread has that discussion:
> 
> http://lore.kernel.org/0-v1-9912f1a11620+2a-fwctl_jgg@nvidia.com
> 
> I do not want to speak for others on the saliency of their points, all I
> can say is that the contrary positions have so far not moved me to drop
> consideration of fwctl for CXL.

I was resisting rat holing. Oh well...

For a 'subset' of CXL.  There are a wide range of controls that are highly
destructive, potentially to other hosts (simplest one is a command that
will surprise remove someone else's memory). For those I'm not sure
fwctl gets us anywhere - but we still need a solution (Subject to
config gates etc as typically this is BMCs not hosts).
Maybe fwctl eventually ends up with levels of 'safety' (beyond the
current read vs write vs write_full, or maybe those are enough).

Complexities such as message tunneling to multiple components are also
going to be fun, but we want the non destructive bits of those to work
as part of the safe set, so we can get telemetry from downstream devices.

Good to cover the debug and telemetry usecase, but it still leaves us with
gaping holes were we need to solve the permissions problem, perhaps that
is layered on top of fwctl, perhaps something else is needed.

So if fwctl is adopted, I do want the means to use it for the highly
destructive stuff as well!  Maybe that's a future discussion.


> 
> Where CXL has a Command Effects Log that is a reasonable protocol for
> making decisions about opaque command codes, and that CXL already has a
> few years of experience with the commands that *do* need a Linux-command
> wrapper.

Worth asking if this will incorporate unknown but not vendor defined
commands.  There is a long tail of stuff in the spec we haven't caught up
with yet.  Or you thinking keep this for the strictly vendor defined stuff?

> 
> Some open questions from that thread are: what does it mean for the fate
> of a proposal if one subsystem Acks the ABI and another Naks it for a
> device that crosses subsystem functionality? Would a cynical hardware
> response just lead to plumbing an NVME admin queue, or CXL mailbox to
> get device-specific commands past another subsystem's objection?
> 
> My reconsideration of the "debug-build only" policy for CXL
> device-specific commands was influenced by a conversation with a distro
> developer where they asserted, paraphrasing: "at what point is a device
> vendor incentivized to ship an out-of-tree module just to restore their
> passthrough functionality?. At that point upstream has lost out on
> collaboration and distro kernel ABI has gained another out-of-tree
> consumer."
> 
> So the tension is healthy, but it has diminishing returns past a certain
> point.
> 

Powered by blists - more mailing lists