[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID:
<BN8PR15MB25133D6B2BC61C81408D6FE899D22@BN8PR15MB2513.namprd15.prod.outlook.com>
Date: Fri, 14 Mar 2025 14:53:40 +0000
From: Bernard Metzler <BMT@...ich.ibm.com>
To: Leon Romanovsky <leon@...nel.org>,
Nikolay Aleksandrov
<nikolay@...abrica.net>
CC: "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
"shrijeet@...abrica.net" <shrijeet@...abrica.net>,
"alex.badea@...sight.com"
<alex.badea@...sight.com>,
"eric.davis@...adcom.com"
<eric.davis@...adcom.com>,
"rip.sohan@....com" <rip.sohan@....com>,
"dsahern@...nel.org" <dsahern@...nel.org>,
"roland@...abrica.net"
<roland@...abrica.net>,
"winston.liu@...sight.com"
<winston.liu@...sight.com>,
"dan.mihailescu@...sight.com"
<dan.mihailescu@...sight.com>,
Kamal Heib <kheib@...hat.com>,
"parth.v.parikh@...sight.com" <parth.v.parikh@...sight.com>,
Dave Miller
<davem@...hat.com>,
"ian.ziemba@....com" <ian.ziemba@....com>,
"andrew.tauferner@...nelisnetworks.com"
<andrew.tauferner@...nelisnetworks.com>,
"welch@....com" <welch@....com>,
"rakhahari.bhunia@...sight.com" <rakhahari.bhunia@...sight.com>,
"kingshuk.mandal@...sight.com" <kingshuk.mandal@...sight.com>,
"linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
"kuba@...nel.org"
<kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>,
Jason Gunthorpe
<jgg@...dia.com>
Subject: RE: [RFC PATCH 00/13] Ultra Ethernet driver introduction
> -----Original Message-----
> From: Leon Romanovsky <leon@...nel.org>
> Sent: Wednesday, March 12, 2025 4:11 PM
> To: Nikolay Aleksandrov <nikolay@...abrica.net>
> Cc: netdev@...r.kernel.org; shrijeet@...abrica.net;
> alex.badea@...sight.com; eric.davis@...adcom.com; rip.sohan@....com;
> dsahern@...nel.org; Bernard Metzler <BMT@...ich.ibm.com>;
> roland@...abrica.net; winston.liu@...sight.com;
> dan.mihailescu@...sight.com; Kamal Heib <kheib@...hat.com>;
> parth.v.parikh@...sight.com; Dave Miller <davem@...hat.com>;
> ian.ziemba@....com; andrew.tauferner@...nelisnetworks.com; welch@....com;
> rakhahari.bhunia@...sight.com; kingshuk.mandal@...sight.com; linux-
> rdma@...r.kernel.org; kuba@...nel.org; Paolo Abeni <pabeni@...hat.com>;
> Jason Gunthorpe <jgg@...dia.com>
> Subject: [EXTERNAL] Re: [RFC PATCH 00/13] Ultra Ethernet driver
> introduction
>
> On Wed, Mar 12, 2025 at 04:20:08PM +0200, Nikolay Aleksandrov wrote:
> > On 3/12/25 1:29 PM, Leon Romanovsky wrote:
> > > On Wed, Mar 12, 2025 at 11:40:05AM +0200, Nikolay Aleksandrov wrote:
> > >> On 3/8/25 8:46 PM, Leon Romanovsky wrote:
> > >>> On Fri, Mar 07, 2025 at 01:01:50AM +0200, Nikolay Aleksandrov wrote:
> > [snip]
> > >> Also we have the ephemeral PDC connections>> that come and go as
> > needed. There more such objects coming with more
> > >> state, configuration and lifecycle management. That is why we added a
> > >> separate netlink family to cleanly manage them without trying to fit
> > >> a square peg in a round hole so to speak.
> > >
> > > Yeah, I saw that you are planning to use netlink to manage objects,
> > > which is very questionable. It is slow, unreliable, requires sockets,
> > > needs more parsing logic e.t.c
> > >
> > > To avoid all this overhead, RDMA uses netlink-like ioctl calls, which
> > > fits better for object configurations.
> > >
> > > Thanks
> >
> > We'd definitely like to keep using netlink for control path object
> > management. Also please note we're talking about genetlink family. It is
> > fast and reliable enough for us, very easily extensible,
> > has a nice precise object definition with policies to enforce various
> > limitations, has extensive tooling (e.g. ynl), communication can be
> > monitored in realtime for debugging (e.g. nlmon), has a nice human
> > readable error reporting, gives the ability to easily dump large object
> > groups with filters applied, YAML family definitions and so on.
> > Having sockets or parsing are not issues.
>
> Of course it is issue as netlink relies on Netlink sockets, which means
> that you constantly move your configuration data instead of doing
> standard to whole linux kernel pattern of allocating configuration
> structs in user-space and just providing pointer to that through ioctl
> call.
>
> However, this discussion is premature and as an intro it is worth to
> read this cover letter for how object management is done in RDMA
> subsystem.
>
> https://lore.kernel.org/linux%
> 2Drdma_1501765627-2D104860-2D1-2Dgit-2Dsend-2Demail-2Dmatanb-
> 40mellanox.com_&d=DwIBAg&c=BSDicqBQBDjDI9RkVyTcHQ&r=4ynb4Sj_4MUcZXbhvovE4tY
> SbqxyOwdSiLedP4yO55g&m=U78K-khiLd-
> LLkbuNRzBStNppsXFTXdM7br052fwal1mzxpaOcOSQXCnguAK8t3g&s=U9dQl07fp-
> e9380xjR94fW-UGixoMsoxr5HfXKYggLk&e=
>
Nice old stuff. Often history teaches us something. 😉
I assume the correct way forward is to first clarify the
structure of all user-visible objects that need to be
created/controlled/destroyed, and to route them through
this interface. Some will require extensions to given objects,
some may be new, some will be as-is. rdma_netlink will probably
be the right interface to look at for job control.
Best,
Bernard.
> Thanks
>
> >
> > Cheers,
> > Nik
> >
> >
Powered by blists - more mailing lists