netdev - RE: [RFC PATCH 00/13] Ultra Ethernet driver introduction

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID:
 <DM6PR12MB4313C577073C5ABE1507401BBDB52@DM6PR12MB4313.namprd12.prod.outlook.com>
Date: Tue, 8 Apr 2025 04:40:11 +0000
From: Sean Hefty <shefty@...dia.com>
To: "Ziemba, Ian" <ian.ziemba@....com>, Jason Gunthorpe <jgg@...dia.com>
CC: Bernard Metzler <BMT@...ich.ibm.com>, Roland Dreier
	<roland@...abrica.net>, Nikolay Aleksandrov <nikolay@...abrica.net>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>, "shrijeet@...abrica.net"
	<shrijeet@...abrica.net>, "alex.badea@...sight.com"
	<alex.badea@...sight.com>, "eric.davis@...adcom.com"
	<eric.davis@...adcom.com>, "rip.sohan@....com" <rip.sohan@....com>,
	"dsahern@...nel.org" <dsahern@...nel.org>, "winston.liu@...sight.com"
	<winston.liu@...sight.com>, "dan.mihailescu@...sight.com"
	<dan.mihailescu@...sight.com>, Kamal Heib <kheib@...hat.com>,
	"parth.v.parikh@...sight.com" <parth.v.parikh@...sight.com>, Dave Miller
	<davem@...hat.com>, "andrew.tauferner@...nelisnetworks.com"
	<andrew.tauferner@...nelisnetworks.com>, "welch@....com" <welch@....com>,
	"rakhahari.bhunia@...sight.com" <rakhahari.bhunia@...sight.com>,
	"kingshuk.mandal@...sight.com" <kingshuk.mandal@...sight.com>,
	"linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>, "kuba@...nel.org"
	<kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>
Subject: RE: [RFC PATCH 00/13] Ultra Ethernet driver introduction

> > There's also the MR relationship:
> >
> > Job <- 1 --- 0..n -> MR <- 0..n --- 1 -> PD
> 
> Current UE memory registration is centered around the libfabric
> FI_MR_ENDPOINT mode which states memory regions are associated with an
> endpoint instead of libfabric domain. For remotely accessible UE MRs, this
> translates to the following.
>
> - Relative Addressing: {job ID, endpoint, RKEY} identifies the MR.
> 
> - Absolute Addressing: {endpoint, RKEY} identifies the MR with optional
>   MR job-ID access control check.
>
> In addition, UE memory registration supports user-defined RKEYs. This enables
> programming model implementations the optimization to use well-known per
> process endpoint RKEYs. For relative addressing, this could result in the same
> RKEY value existing multiple times at the {job ID} level, but only once at the
> {job ID, endpoint} level.
> 
> The UE remote MR relationship would look like:
> 
> Job <- 1 --- 0..n -> EP <- 0..n ---------------------- 1 -> PD
>                       ^                                      ^
>                       |--- 1 --- 0..n -> MR <- 0..n --- 1 ---|

IMO, job support should be transport agnostic and include connection-oriented RDMA transports.

I might have ended up too concise.  Transport header fields identify QP, MR, and job objects.  I think in terms of these minimal mappings:

QPN -> QP
rkey -> MR
(also lkey -> MR)
Job ID -> job

It's an oversimplification.  Obviously, QPN=1 uses other fields.  Maybe Job ID is actually 'pkey'.  Each vendor owns mapping transport fields to SW visible objects.

Libfabric supports both vendor and user selected rkeys.  I don't think this matters to the model.  If rkeys are device unique, rkey -> MR is trivial.  If rkeys are not, that's a feature-complexity trade-off.  Similarly, a vendor may support 1 job per QP or many.  Regardless, I still view the security model as finding a valid tuple:

{PD, QP, MR, Job}

The MR and/or Job may be N/A for any given transfer or configuration.

> > There's discussion on defining this relationship:
> >
> > Job <- 0..n --- 1 -> PD
> >
> > I can't think of a technical reason why that's needed.
> 
> From my UE perspective, I agree. UE needs to share job IDs across processes
> while still having inter-process isolation for things like local memory
> registrations.

I distinguish job instance running on the cluster from an instance of some job object.  Two object instances, identified by some other scope, could result in the same job ID.  (E.g. each process has its own object.) 

I lean towards defining a device specific job object, assuming HW resources may be required.  Job attributes include selected transport and address/ID.

Is this sufficient scope?  If a job maps to HW resources and is used with transport processing, does it need process isolation that comes with pairing it to a PD?  Or is it enough that the job is paired with the QP and, optionally, MR?  Is the creation of a job object always a privileged operation, or is only the act of assigning the job ID privileged?

Thinking through implementation options, since job is specified per transfer, it seems easier to validate the 'jkey' written with a WR shares the same PD of the QP, similar to how an lkey check might work.  One alternative is a jkey list off the QP.  Other options could offload libfabric address vectors.  It's possible vendors may implement this differently, with differences showing up in which MRs are reachable.

- Sean