netdev - RE: [RFC 07/11] Add support for memory registeration verbs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <BLUPR0701MB19225000802B5D226FB4CBBD88F10@BLUPR0701MB1922.namprd07.prod.outlook.com>
Date:   Wed, 14 Sep 2016 10:02:02 +0000
From:   "Kalderon, Michal" <Michal.Kalderon@...ium.com>
To:     Sagi Grimberg <sagi@...mberg.me>,
        Ram Amrani <Ram.Amrani@...gic.com>,
        "dledford@...hat.com" <dledford@...hat.com>,
        "davem@...emloft.net" <davem@...emloft.net>
CC:     "Yuval.Mintz@...gic.com" <Yuval.Mintz@...gic.com>,
        "Ariel.Elior@...gic.com" <Ariel.Elior@...gic.com>,
        "Michal.Kalderon@...gic.com" <Michal.Kalderon@...gic.com>,
        "rajesh.borundia@...gic.com" <rajesh.borundia@...gic.com>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: RE: [RFC 07/11] Add support for memory registeration verbs

> >>> +struct qedr_mr *__qedr_alloc_mr(struct ib_pd *ibpd, int
> >>> +max_page_list_len) {
> >>> +	struct qedr_pd *pd = get_qedr_pd(ibpd);
> >>> +	struct qedr_dev *dev = get_qedr_dev(ibpd->device);
> >>> +	struct qedr_mr *mr;
> >>> +	int rc = -ENOMEM;
> >>> +
> >>> +	DP_VERBOSE(dev, QEDR_MSG_MR,
> >>> +		   "qedr_alloc_frmr pd = %d max_page_list_len= %d\n", pd-
> >>> pd_id,
> >>> +		   max_page_list_len);
> >>> +
> >>> +	mr = kzalloc(sizeof(*mr), GFP_KERNEL);
> >>> +	if (!mr)
> >>> +		return ERR_PTR(rc);
> >>> +
> >>> +	mr->dev = dev;
> >>> +	mr->type = QEDR_MR_FRMR;
> >>> +
> >>> +	rc = init_mr_info(dev, &mr->info, max_page_list_len, 1);
> >>> +	if (rc)
> >>> +		goto err0;
> >>> +
> >>> +	rc = dev->ops->rdma_alloc_tid(dev->rdma_ctx, &mr->hw_mr.itid);
> >>> +	if (rc) {
> >>> +		DP_ERR(dev, "roce alloc tid returned an error %d\n", rc);
> >>> +		goto err0;
> >>> +	}
> >>> +
> >>> +	/* Index only, 18 bit long, lkey = itid << 8 | key */
> >>> +	mr->hw_mr.tid_type = QED_RDMA_TID_FMR;
> >>> +	mr->hw_mr.key = 0;
> >>> +	mr->hw_mr.pd = pd->pd_id;
> >>
> >> Do you have a real MR<->PD association in HW? If so, can you point me
> >> to the code that binds it? If not, any reason not to expose the
> local_dma_lkey?
> >>
> > Yes, we send the pd id to the FW in function qed_rdma_register_tid.
> 
> Right, thanks.
> 
> > In any case, if we didn't have the association in HW Wouldn't the
> > local_dma_lkey be relevant only to dma_mr ? ( code snippet above
> > refers to FMR)
> 
> I was just sticking to a location in the code where you associate MR<->PD...
> 
> The local_dma_lkey is a special key that spans the entire memory space and
> unlike the notorious dma_mr, its not associated with a PD.
> 
> See the code in ib_alloc_pd(), if the device does not support a single device
> local dma lkey, the core allocates a dma mr associated with the pd. If your
> device has such a key, you can save a dma mr allocation for each pd in the
> system.
Managed to miss this. Our device supports such a key, we'll add support.