Date: Tue, 4 Jun 2024 20:05:56 -0700
From: Jakub Kicinski <kuba@...nel.org>
To: Dan Williams <dan.j.williams@...el.com>
Cc: David Ahern <dsahern@...nel.org>, Jason Gunthorpe <jgg@...dia.com>,
 Jonathan Corbet <corbet@....net>, "Itay Avraham" <itayavr@...dia.com>, Leon
 Romanovsky <leon@...nel.org>, <linux-doc@...r.kernel.org>,
 <linux-rdma@...r.kernel.org>, <netdev@...r.kernel.org>, Paolo Abeni
 <pabeni@...hat.com>, Saeed Mahameed <saeedm@...dia.com>, Tariq Toukan
 <tariqt@...dia.com>, Andy Gospodarek <andrew.gospodarek@...adcom.com>, Aron
 Silverton <aron.silverton@...cle.com>, Christoph Hellwig
 <hch@...radead.org>, Jiri Pirko <jiri@...dia.com>, Leonid Bloch
 <lbloch@...dia.com>, Leon Romanovsky <leonro@...dia.com>,
 <linux-cxl@...r.kernel.org>, <patches@...ts.linux.dev>
Subject: Re: [PATCH 0/8] Introduce fwctl subystem

On Tue, 4 Jun 2024 16:56:57 -0700 Dan Williams wrote:
> Jakub Kicinski wrote:
> [..]
> > I don't begrudge anyone building proprietary options, but leave
> > upstream out of it.  
> 
> So I am of 2 minds here. In general, how is upstream benefited by
> requiring every vendor command to be wrapped by a Linux command?
> [...]

Thanks for sharing the CXL experience and your perspective.
Also for trying to frame the discussion in a useful way,
although I have little faith that it will help :( Fingers crossed?

> * Integrity: Subsystem has a responsibility to meet kernel-lockdown
>   expectations:
> 
>   Distros and system owners need to be assured that root's ability to
>   modify the running kernel image are mitigated. For CXL there are 2 ways
>   to do this, require Linux wrapper commands for all the low level
>   commands (status quo), or a new trust the device to publish which
>   commands have user data effects in something CXL calls the "Command
>   Effects Log". In that "trust Command Effects" scenario the kernel still
>   has no idea what the command is actually doing, but it can at least
>   assert that the device does not claim that the command changes the
>   contents of system-memory. Now, you might say, "the device can just
>   lie", but that betrays a conceit of the kernel restriction. A device
>   could lie that a Linux wrapped command when passed certain payloads does
>   not in turn proxy to a restricted command. So at some point there is
>   almost always an out-of-tree way to get around the kernel restriction,
>   so the question is are we better off giving a blessed path or force
>   vendors into ugly out-of-tree workarounds?

The integrity thing is a double edge sword, so I don't have much to say
here. If we take a few wrong turns we'll wrap the vendor commands with
crypto and then the vendor can control which commands you get to run ;)
Obviously I'm joking, and not saying that the intent of the current
series! But its about as realistic as "this will only be used for truly
vendor specific things".

> * Introspection / validation: Subsystem community needs to be able to
>   audit behavior after the fact.
> 
>   To me this means even if the kernel is letting a command through based
>   on the stated Command Effect of "Configuration Change after Cold Reset"
>   upstream community has a need to be able to read the vendor
>   specification for that command. I.e. commands might be vendor-specific,
>   but never vendor-private. I see this as similar to the requirement for
>   open source userspace for sophisticated accelerators.

That sounds pretty CXL specific, and IIUC unrealistic.
You assume you have some specification to consult, while this discussion
has been going for over a year now, and I can't get the vendors to share
what those turntables they so desperately need to tweak are.

> * Collaboration: open standards support open driver maintenance.
> 
>   Without standards we end up with awkward situations like Confidential
>   Computing where every vendor races to implement the same functionality
>   in arbitrarily different and vendor specific ways.
> 
>   For CXL devices, and I believe the devices fwctl is targeting, there
>   are a whole class of commands for vendor specific configuration and
>   debug. Commands that the kernel really need not worry about.
> 
>   Some subsystems may want to allow high-performance science experiments
>   like what NVMe allows, but it seems worth asking the question if
>   standardizing device configuration and debug is really the best use of
>   upstream's limited time?

No, but it's not about science experiments, really. It's about
production features. The effort of implementing something properly
upstream is high. I cost time and money to get the right caliber of
people and let them go thru the revisions. I lack confidence that
merging fwctl will not negatively impact motivation for companies to
pay off our accrued technical debt. While all they need is "this simple
little feature". And before competition wins the customer. It's a race
to the bottom.

>   One of the release valves in the CXL space is openly specified
>   commands with opaque payloads, like "Read Vendor Debug Log". That is
>   clear what it does, likely a payload the kernel need never worry
>   about, and the "Command Effects" is empty. However, going forward there
>   is a new class of commands called "Set/Get Feature" that allow a wide
>   range of vendor toggles to be deployed which will need an upstream
>   response for the driver policy to vendor-specific "Features".
>
> So if fwctl, or something like it, can strike a balance of enforcing
> integrity and introspection while encouraging collaboration on the
> aspects that are worth upstream collaboration, I think that is a
> conversation worth having.

I presume you were trying to underscore that the decision is unavoidably
a trade off, which is true. But I don't follow the exact formulation.
Is fwctl helping integrity or collaboration? If we assume use of vendor
tools is unavoidable, then I guess integrity? I really can't see how it
helps collaboration when everyone ships their custom tool set.

Back to the tradeoff. For networking, which is a _very_ mature subsystem
with a ton of standards the need to do "vendor specific things" is
marginal. The downside of the loss of an "upstream advantage" is obvious.
We need to take such decisions on subsystem by subsystem basis.
You should be able to draw the lines differently for CXL than how we
draw them for TCP/IP.

On the technical level the discussion can't go very far, because I'd
like to hear actual user problems. But I can't even get a list of those
infamous thousands of knobs :|

Powered by blists - more mailing lists