[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID:
<DM6PR12MB431345D07D958CF0B784AE0EBDA62@DM6PR12MB4313.namprd12.prod.outlook.com>
Date: Wed, 26 Mar 2025 17:39:52 +0000
From: Sean Hefty <shefty@...dia.com>
To: Jason Gunthorpe <jgg@...dia.com>
CC: Bernard Metzler <BMT@...ich.ibm.com>, Roland Dreier
<roland@...abrica.net>, Nikolay Aleksandrov <nikolay@...abrica.net>,
"netdev@...r.kernel.org" <netdev@...r.kernel.org>, "shrijeet@...abrica.net"
<shrijeet@...abrica.net>, "alex.badea@...sight.com"
<alex.badea@...sight.com>, "eric.davis@...adcom.com"
<eric.davis@...adcom.com>, "rip.sohan@....com" <rip.sohan@....com>,
"dsahern@...nel.org" <dsahern@...nel.org>, "winston.liu@...sight.com"
<winston.liu@...sight.com>, "dan.mihailescu@...sight.com"
<dan.mihailescu@...sight.com>, Kamal Heib <kheib@...hat.com>,
"parth.v.parikh@...sight.com" <parth.v.parikh@...sight.com>, Dave Miller
<davem@...hat.com>, "ian.ziemba@....com" <ian.ziemba@....com>,
"andrew.tauferner@...nelisnetworks.com"
<andrew.tauferner@...nelisnetworks.com>, "welch@....com" <welch@....com>,
"rakhahari.bhunia@...sight.com" <rakhahari.bhunia@...sight.com>,
"kingshuk.mandal@...sight.com" <kingshuk.mandal@...sight.com>,
"linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>, "kuba@...nel.org"
<kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>
Subject: RE: [RFC PATCH 00/13] Ultra Ethernet driver introduction
> > > Like I said already, I think Job needs to be a first class RDMA
> > > object that is used by all transports that have job semantics.
> >
> > How do you handle or expose device specific resource allocations or
> > restrictions, which may be needed? Should a kernel 'RDMA job manager'
> > abstract device level resources?
> >
> > Consider a situation where a MR or MW should only be accessible by a
> > specific job. When the MR is created, the device specific job
> > resource may be needed. Should drivers need to query the job manager
> > to map some global object to a device specific resource?
>
> I imagine for cases like that the job would be linked to the PD and then MR ->
> PD -> Job.
>
> The kernel side would create any HW object for the job when the PD is created
> for a specific HW device.
>
> The PD security semantic for the MR would be a little bit different in that the
> PD is more like a shared PD.
The PD is a problem, as it's not a transport function. It's a hardware implementation component; one which may NOT exist for a UEC NIC. (I know there are NICs which do not implement PDs and have secure RDMA transfers.) I have a proposal to rework/redefine PDs to support a more general model, which I think will work for NICs that need a PD and ones that don't. It can support MR -> PD -> Job, but I considered the PD -> job relationship as 1 to many. I can't immediately think of a reason why a 1:1 'job-based PD' wouldn't work in theory.
It's challenging in that a UET endpoint (QP) may communicate with multiple jobs, and a MR may be accessible by a single job, all jobs, or only a few.
Basically, the RDMA PD model forces a HW implementation. Some, but not all, NICs will implement this. But in general, there's not a clean {PD, QP, MR, job} relationship.
- Sean
Powered by blists - more mailing lists