netdev - Re: [RFC PATCH 00/13] Ultra Ethernet driver introduction

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CALgUMKhB7nZkU0RtJJRtcHFm2YVmahUPCQv2XpTwZw=PaaiNHg@mail.gmail.com>
Date: Mon, 24 Mar 2025 13:22:13 -0700
From: Roland Dreier <roland@...abrica.net>
To: Jason Gunthorpe <jgg@...dia.com>
Cc: Nikolay Aleksandrov <nikolay@...abrica.net>, netdev@...r.kernel.org, shrijeet@...abrica.net, 
	alex.badea@...sight.com, eric.davis@...adcom.com, rip.sohan@....com, 
	dsahern@...nel.org, bmt@...ich.ibm.com, winston.liu@...sight.com, 
	dan.mihailescu@...sight.com, kheib@...hat.com, parth.v.parikh@...sight.com, 
	davem@...hat.com, ian.ziemba@....com, andrew.tauferner@...nelisnetworks.com, 
	welch@....com, rakhahari.bhunia@...sight.com, kingshuk.mandal@...sight.com, 
	linux-rdma@...r.kernel.org, kuba@...nel.org, pabeni@...hat.com
Subject: Re: [RFC PATCH 00/13] Ultra Ethernet driver introduction

Hi Jason,

I think we were not clear on the overall discussion and so we are much
closer to agreement than you might think, see below.

 > I was away while this discussion happened so I've gone through and
 > read the threads, looked at the patches and I don't think I've changed
 > my view since I talked to Enfabrica privately on this topic almost a
 > year ago.

First, want to clarify that this patchset is collaborative development
within the overall Ultra Ethernet Consortium. That's not to take away
from the large effort that Nik from Enfabrica put into this but simply
to give a little more context.

 > I do not agree with creating a new subsystem (or whatever you are
 > calling drivers/ultraeth) for a single RDMA protocol and see nothing
 > new here to change my mind. I would likely NAK the direction I see in
 > this RFC, as I have other past attempts to build RDMA HW interfaces
 > outside of the RDMA subystem.

UEC is definitely not trying to create anything new beyond adding
support for Ultra Ethernet. By far the bulk of this patchset is adding
a software model of the specific Ultra Ethernet transport's protocol /
packet handling, and that code is currently in drivers/ultraeth. I
don't feel that pathnames are particularly important, and we could
move the code to something like drivers/infiniband/ultraeth, but that
seems a bit silly. But certainly we are open to suggestions.

 >    So, this RFC is woefully incomplete. I think you greatly underestimate
 >    how much work you are looking at to duplicate and re-invent the
 >    existing RDMA infrastructure. Frankly I'm not even sure why you
 >    sent this RFC when it doesn't show enough to even evaluate..

Total agreement that this RFC is incomplete! We are trying to get
something out early, exactly to get the discussion started and agree
on the best way to add kernel support for UE.

To be clear - we are not trying to reinvent or bypass uverbs, and
there is complete agreement within UEC that we should reuse the uverbs
infrastructure so that we get the advantages of solid, mature
mechanisms for memory pinning, resource tracking / cleanup ordering,
etc.

With that said, Ultra Ethernet devices likely will not have interfaces
that map well onto QPs, MRs, etc. so we will be sending patches to
drivers/infiniband/uverbs* that generalize things to allow "struct
ib_device" objects that do not implement "required verbs."

 >    Ie you said:
 >
 >     > I should've been more specific - it is not an issue for UEC and the way
 >     > our driver's netlink API is designed. We fully understand the pros and
 >     > cons of our approach.
 >
 >    Which is exactly the kind of narrow thinking that creates long term
 >    trouble in uAPI design. Do your choices actually work for *ALL*
 >    future HW designs and others drivers not just "our drivers
 >    netlink"? I think not.
 >
 >    Given UE spec doesn't even have something pretending to be a
 >    kernel/user interface standard I think we will see an extreme
 >    variety of HW implementations here.

I think the netlink API and job handling overall is the area where the
most discussion is probably required. UE is somewhat novel in
elevating the concept of a "job" to a standard object with specific
properties that determine the values in packet headers. But I'm open
to making "job" a top-level RDMA object... I guess the idea would be
to define an interface for creating a new type of "job FD" with a
standard ABI for setting properties?

 - Roland