[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250526131938.GB9786@nvidia.com>
Date: Mon, 26 May 2025 10:19:38 -0300
From: Jason Gunthorpe <jgg@...dia.com>
To: Abhijit Gangurde <abhijit.gangurde@....com>
Cc: shannon.nelson@....com, brett.creeley@....com, davem@...emloft.net,
edumazet@...gle.com, kuba@...nel.org, pabeni@...hat.com,
corbet@....net, leon@...nel.org, andrew+netdev@...n.ch,
allen.hubbe@....com, nikhil.agarwal@....com,
linux-rdma@...r.kernel.org, netdev@...r.kernel.org,
linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2 00/14] Introduce AMD Pensando RDMA driver
On Thu, May 22, 2025 at 04:58:19PM +0530, Abhijit Gangurde wrote:
> > @@ -1231,6 +1257,7 @@ void ionic_destroy_rdma_admin(struct ionic_ibdev *dev)
> > }
> > if (dev->eq_vec) {
> > + // Locking? Add a lockdep assertion if caller is holding the lock
> > while (dev->lif_cfg.eq_count > 0) {
> > eq = dev->eq_vec[--dev->lif_cfg.eq_count];
> > ionic_destroy_eq(eq);
> I don't think there is a need for the lock here because the device is
> unregistered and the queues are all stopped.
Add a comment then
> > @@ -887,6 +893,8 @@ static struct ib_mr *ionic_get_dma_mr(struct ib_pd *ibpd, int access)
> > if (!mr)
> > return ERR_PTR(-ENOMEM);
> > + // This seems strange, shouldn't this do something? If you don't support an all address MR then don't define this op.
> > +
> > return &mr->ibmr;
> > }
> From hardware lkey zero is reserved as a local dma lkey for all address MR.
> I would make it more explicit as mr.ibmr.lkey = IONIC_DMA_LKEY (same for
> RKEY) with that defined to be zero.
Yeah, that's a lot clearer
> > @@ -1454,11 +1466,15 @@ static int ionic_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
> > static bool pd_local_privileged(struct ib_pd *pd)
> > {
> > + /* That isn't how it works, only the lkey get_dma_mr() returns is
> > + special and must be used on any WRs that require it. WRs refering to any
> > + other lkeys must behave normally. */
> > return !pd->uobject;
> > }
> > static bool pd_remote_privileged(struct ib_pd *pd)
> > {
> > + /* Same comment, except about rkeys now. */
> > return pd->flags & IB_PD_UNSAFE_GLOBAL_RKEY;
> > }
> This is how we allow the qp to use the dma lkey. If the qp is a kernel
> space qp (its pd has no uobject) then we allow use of the dma lkey by that
> qp. We do not allow use of dma lkey by user qps. If the pd flags has the
> unsafe rkey flag, then we also allow the qp use it for remote access.
OK, so this is just about permissions because you don't attach the 0
l/rkey to a PD? Add a comment in the ionic_get_dma_mr() that the QP
needs special flags to be allowed to use the 0 l/rkey.
Jason
Powered by blists - more mailing lists