| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID:
<SI2P153MB04416E4B445A988965398AB5BB51A@SI2P153MB0441.APCP153.PROD.OUTLOOK.COM>
Date: Fri, 9 Jun 2023 01:15:07 +0000
From: Wei Hu <weh@...rosoft.com>
To: Long Li <longli@...rosoft.com>, "netdev@...r.kernel.org"
<netdev@...r.kernel.org>, "linux-hyperv@...r.kernel.org"
<linux-hyperv@...r.kernel.org>, "linux-rdma@...r.kernel.org"
<linux-rdma@...r.kernel.org>, Ajay Sharma <sharmaajay@...rosoft.com>,
"jgg@...pe.ca" <jgg@...pe.ca>, "leon@...nel.org" <leon@...nel.org>, KY
Srinivasan <kys@...rosoft.com>, Haiyang Zhang <haiyangz@...rosoft.com>,
"wei.liu@...nel.org" <wei.liu@...nel.org>, Dexuan Cui <decui@...rosoft.com>,
"davem@...emloft.net" <davem@...emloft.net>, "edumazet@...gle.com"
<edumazet@...gle.com>, "kuba@...nel.org" <kuba@...nel.org>,
"pabeni@...hat.com" <pabeni@...hat.com>, "vkuznets@...hat.com"
<vkuznets@...hat.com>, "ssengar@...ux.microsoft.com"
<ssengar@...ux.microsoft.com>, "shradhagupta@...ux.microsoft.com"
<shradhagupta@...ux.microsoft.com>
Subject: RE: [PATCH v2 1/1] RDMA/mana_ib: Add EQ interrupt support to mana ib
driver.
> -----Original Message-----
> From: Long Li <longli@...rosoft.com>
> Sent: Friday, June 9, 2023 1:48 AM
> To: Wei Hu <weh@...rosoft.com>; netdev@...r.kernel.org; linux-
> hyperv@...r.kernel.org; linux-rdma@...r.kernel.org; Ajay Sharma
> <sharmaajay@...rosoft.com>; jgg@...pe.ca; leon@...nel.org; KY
> Srinivasan <kys@...rosoft.com>; Haiyang Zhang <haiyangz@...rosoft.com>;
> wei.liu@...nel.org; Dexuan Cui <decui@...rosoft.com>;
> davem@...emloft.net; edumazet@...gle.com; kuba@...nel.org;
> pabeni@...hat.com; vkuznets@...hat.com; ssengar@...ux.microsoft.com;
> shradhagupta@...ux.microsoft.com
> Subject: RE: [PATCH v2 1/1] RDMA/mana_ib: Add EQ interrupt support to
> mana ib driver.
>
> > Subject: RE: [PATCH v2 1/1] RDMA/mana_ib: Add EQ interrupt support to
> > mana ib driver.
> >
> >
> >
> > > -----Original Message-----
> > > Subject: RE: [PATCH v2 1/1] RDMA/mana_ib: Add EQ interrupt support
> > > to mana ib driver.
> > >
> > > > Subject: [PATCH v2 1/1] RDMA/mana_ib: Add EQ interrupt support to
> > > > mana ib driver.
> > > >
> > > > Add EQ interrupt support for mana ib driver. Allocate EQs per
> > > > ucontext to receive interrupt. Attach EQ when CQ is created. Call
> > > > CQ interrupt handler when completion interrupt happens. EQs are
> > > > destroyed when
> > > ucontext is deallocated.
> > > >
> > > > The change calls some public APIs in mana ethernet driver to
> > > > allocate EQs and other resources. Ehe EQ process routine is also
> > > > shared by mana ethernet and mana ib drivers.
> > > >
> > > > Co-developed-by: Ajay Sharma <sharmaajay@...rosoft.com>
> > > > Signed-off-by: Ajay Sharma <sharmaajay@...rosoft.com>
> > > > Signed-off-by: Wei Hu <weh@...rosoft.com>
> > > > ---
> > > >
> > > > v2: Use ibdev_dbg to print error messages and return -ENOMEN
> > > > when kzalloc fails.
> > > >
> > > > drivers/infiniband/hw/mana/cq.c | 32 ++++-
> > > > drivers/infiniband/hw/mana/main.c | 87 ++++++++++++
> > > > drivers/infiniband/hw/mana/mana_ib.h | 4 +
> > > > drivers/infiniband/hw/mana/qp.c | 90 +++++++++++-
> > > > .../net/ethernet/microsoft/mana/gdma_main.c | 131 ++++++++++---
> ----
> > -
> > > > drivers/net/ethernet/microsoft/mana/mana_en.c | 1 +
> > > > include/net/mana/gdma.h | 9 +-
> > > > 7 files changed, 290 insertions(+), 64 deletions(-)
> > > >
> > > > diff --git a/drivers/infiniband/hw/mana/cq.c
> > > > b/drivers/infiniband/hw/mana/cq.c index
> d141cab8a1e6..3cd680e0e753
> > > > 100644
> > > > --- a/drivers/infiniband/hw/mana/cq.c
> > > > +++ b/drivers/infiniband/hw/mana/cq.c
> > > > @@ -12,13 +12,20 @@ int mana_ib_create_cq(struct ib_cq *ibcq,
> > > > const struct ib_cq_init_attr *attr,
> > > > struct ib_device *ibdev = ibcq->device;
> > > > struct mana_ib_create_cq ucmd = {};
> > > > struct mana_ib_dev *mdev;
> > > > + struct gdma_context *gc;
> > > > + struct gdma_dev *gd;
> > > > int err;
> > > >
> > > > mdev = container_of(ibdev, struct mana_ib_dev, ib_dev);
> > > > + gd = mdev->gdma_dev;
> > > > + gc = gd->gdma_context;
> > > >
> > > > if (udata->inlen < sizeof(ucmd))
> > > > return -EINVAL;
> > > >
> > > > + cq->comp_vector = attr->comp_vector > gc->max_num_queues ?
> > > > + 0 : attr->comp_vector;
> > > > +
> > > > err = ib_copy_from_udata(&ucmd, udata, min(sizeof(ucmd), udata-
> > > > >inlen));
> > > > if (err) {
> > > > ibdev_dbg(ibdev,
> > > > @@ -69,11 +76,32 @@ int mana_ib_destroy_cq(struct ib_cq *ibcq,
> > > > struct ib_udata *udata)
> > > > struct mana_ib_cq *cq = container_of(ibcq, struct mana_ib_cq, ibcq);
> > > > struct ib_device *ibdev = ibcq->device;
> > > > struct mana_ib_dev *mdev;
> > > > + struct gdma_context *gc;
> > > > + struct gdma_dev *gd;
> > > > +
> > > >
> > > > mdev = container_of(ibdev, struct mana_ib_dev, ib_dev);
> > > > + gd = mdev->gdma_dev;
> > > > + gc = gd->gdma_context;
> > > >
> > > > - mana_ib_gd_destroy_dma_region(mdev, cq->gdma_region);
> > > > - ib_umem_release(cq->umem);
> > > > +
> > > > +
> > > > + if (atomic_read(&ibcq->usecnt) == 0) {
> > > > + mana_ib_gd_destroy_dma_region(mdev, cq->gdma_region);
> > >
> > > Need to check if this function fails. The following code will call
> > > kfree(gc-
> > > >cq_table[cq->id]), it's possible that IRQ is happening at the same
> > > >time if CQ
> > > is not destroyed.
> > >
> >
> > Sure. Will update.
> >
> > > > + ibdev_dbg(ibdev, "freeing gdma cq %p\n", gc->cq_table[cq-
> > > >id]);
> > > > + kfree(gc->cq_table[cq->id]);
> > > > + gc->cq_table[cq->id] = NULL;
> > > > + ib_umem_release(cq->umem);
> > > > + }
> > > >
> > > > return 0;
> > > > }
> > > > +
> > > > +void mana_ib_cq_handler(void *ctx, struct gdma_queue *gdma_cq) {
> > > > + struct mana_ib_cq *cq = ctx;
> > > > + struct ib_device *ibdev = cq->ibcq.device;
> > > > +
> > > > + ibdev_dbg(ibdev, "Enter %s %d\n", __func__, __LINE__);
> > >
> > > This debug message seems overkill?
> > >
> > > > + cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context); }
> > > > diff --git a/drivers/infiniband/hw/mana/main.c
> > > > b/drivers/infiniband/hw/mana/main.c
> > > > index 7be4c3adb4e2..e4efbcaed10e 100644
> > > > --- a/drivers/infiniband/hw/mana/main.c
> > > > +++ b/drivers/infiniband/hw/mana/main.c
> > > > @@ -143,6 +143,81 @@ int mana_ib_dealloc_pd(struct ib_pd *ibpd,
> > > > struct ib_udata *udata)
> > > > return err;
> > > > }
> > > >
> > > > +static void mana_ib_destroy_eq(struct mana_ib_ucontext *ucontext,
> > > > + struct mana_ib_dev *mdev) {
> > > > + struct gdma_context *gc = mdev->gdma_dev->gdma_context;
> > > > + struct ib_device *ibdev = ucontext->ibucontext.device;
> > > > + struct gdma_queue *eq;
> > > > + int i;
> > > > +
> > > > + if (!ucontext->eqs)
> > > > + return;
> > > > +
> > > > + for (i = 0; i < gc->max_num_queues; i++) {
> > > > + eq = ucontext->eqs[i].eq;
> > > > + if (!eq)
> > > > + continue;
> > > > +
> > > > + mana_gd_destroy_queue(gc, eq);
> > > > + }
> > > > +
> > > > + kfree(ucontext->eqs);
> > > > + ucontext->eqs = NULL;
> > > > +
> > > > + ibdev_dbg(ibdev, "destroyed eq's count %d\n", gc-
> > > >max_num_queues); }
> > >
> > > Will gc->max_num_queues change after destroying a EQ?
> > >
> >
> > I think it will not change. Also the compiler might optimize the code
> > to just read the value once and store it in a register.
> >
> > Thanks,
> > Wei
>
> This message is confusing. How about changing it to " destroyed eq.
> Maximum count %d", or just remove the count as it's not informational.
>
> Long
Sure. I will remove the count from the message.
Thanks,
Wei
Powered by blists - more mailing lists