linux-kernel - Re: [PATCH rdma-next 1/1] RDMA/mana_ib: indicate that inline data is not supported

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID:
 <PAXPR83MB0559FD4684B40F51A67D6AC9B4AD2@PAXPR83MB0559.EURPRD83.prod.outlook.com>
Date: Fri, 19 Jul 2024 10:51:58 +0000
From: Konstantin Taranov <kotaranov@...rosoft.com>
To: Jason Gunthorpe <jgg@...dia.com>
CC: Leon Romanovsky <leon@...nel.org>, Konstantin Taranov
	<kotaranov@...ux.microsoft.com>, Wei Hu <weh@...rosoft.com>,
	"sharmaajay@...rosoft.com" <sharmaajay@...rosoft.com>, Long Li
	<longli@...rosoft.com>, "linux-rdma@...r.kernel.org"
	<linux-rdma@...r.kernel.org>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>
Subject: Re: [PATCH rdma-next 1/1] RDMA/mana_ib: indicate that inline data is
 not supported

> > > > > > Yes, you are. If user asked for specific functionality
> > > > > > (max_inline_data != 0) and your device doesn't support it, you
> > > > > > should
> > > return an error.
> > > > > >
> > > > > > pvrdma, mlx4 and rvt are not good examples, they should return
> > > > > > an error as well, but because of being legacy code, we won't change
> them.
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > >
> > > > > I see. So I guess we can return a larger value, but not smaller. Right?
> > > > > I will send v2 that fails QP creation then.
> > > > >
> > > > > In this case, may I submit a patch to rdma-core that queries
> > > > > device caps before trying to create a qp in rdma_client.c and
> > > > > rdma_server.c? As that code violates what you described.
> > > >
> > > > Let's ask Jason, why is that? Do we allow to ignore max_inline_data?
> > > >
> > > > librdmacm/examples/rdma_client.c
> > > >   63         memset(&attr, 0, sizeof attr);
> > > >   64         attr.cap.max_send_wr = attr.cap.max_recv_wr = 1;
> > > >   65         attr.cap.max_send_sge = attr.cap.max_recv_sge = 1;
> > > >   66         attr.cap.max_inline_data = 16;
> > > >   67         attr.qp_context = id;
> > > >   68         attr.sq_sig_all = 1;
> > > >   69         ret = rdma_create_ep(&id, res, NULL, &attr);
> > > >   70         // Check to see if we got inline data allowed or not
> > > >   71         if (attr.cap.max_inline_data >= 16)
> > > >   72                 send_flags = IBV_SEND_INLINE;
> > > >   73         else
> > > >   74                 printf("rdma_client: device doesn't support
> > > IBV_SEND_INLINE, "
> > > >   75                        "using sge sends\n");
> > >
> > > I think the idea expressed in this code is that if max_inline_data
> > > requested too much it would be limited to the device capability.
> > >
> > > ie qp creation should limit the requests values to what the HW can
> > > do, similar to how entries and other work.
> > >
> > > If the HW has no support it should return - for max_inline_data not
> > > an error, I guess?
> >
> > Yes, this code implies that max_inline_data can be ignored at creation,
> while the manual of ibv_create_qp says:
> > "The function ibv_create_qp() will update the qp_init_attr->cap struct
> > with the actual QP values of the QP that was created; the values will
> > be **greater than or equal to** the values requested."
> 
> Ah, well that seems to be some misunderstandings then, yes.
> 
> > I see two options:
> > 1) Remove code from rdma examples that rely on ignoring max_inline; add
> a warning to libibverbs when drivers ignore that value.
> > 2) Add to manual that max_inline_data might be ignored by drivers; and
> allow my current patch that ignores max_inline_data in mana_ib.
> 
> I don't know, what do the majority of drivers do? If enough are already doing
> 1 then lets force everyone into 1, otherwise we have to document 2.
> 
> And a pyverbs test should be added to cover this weirdness

I quickly read create_qp code of all providers and it seems that max_inline_data is ignored by hw/pvrdma and sw/rvt.
Other providers fail the creation when they cannot satisfy the inline_data cap.
Some drivers ignore it for GSI, but I think it is reasonable. 

Then I guess the option 1 is better. Regarding pyverbs, should I add a test for the option 1?
If yes, what should it test?

> 
> Jason