[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20201002003218.GB1286456@nvidia.com>
Date: Thu, 1 Oct 2020 21:32:18 -0300
From: Jason Gunthorpe <jgg@...dia.com>
To: Leon Romanovsky <leon@...nel.org>
CC: Doug Ledford <dledford@...hat.com>,
Leon Romanovsky <leonro@...dia.com>,
Ariel Elior <aelior@...vell.com>,
Avihai Horon <avihaih@...dia.com>,
<linux-kernel@...r.kernel.org>, <linux-rdma@...r.kernel.org>,
Michal Kalderon <mkalderon@...vell.com>
Subject: Re: [PATCH rdma-next v3 0/4] Query GID table API
On Wed, Sep 23, 2020 at 07:50:11PM +0300, Leon Romanovsky wrote:
> When an application is not using RDMA CM and if it is using multiple RDMA
> devices with one or more RoCE ports, finding the right GID table entry is
> a long process.
>
> For example, with two RoCE dual-port devices in a system, when IP
> failover is used between two RoCE ports, searching a suitable GID
> entry for a given source IP, matching netdevice of given RoCEv1/v2 type
> requires iterating over all 4 ports * 256 entry GID table.
>
> Even though the best first match GID table for given criteria is used,
> when the matching entry is on the 4th port, it requires reading
> 3 ports * 256 entries * 3 files (GID, netdev, type) = 2304 files.
>
> The GID table needs to be referred on every QP creation during IP
> failover on other netdevice of an RDMA device.
>
> In an alternative approach, a GID cache may be maintained and updated on
> GID change event was reported by the kernel. However, it comes with below
> two limitations:
> (a) Maintain a thread per application process instance to listen and update
> the cache.
> (b) Without the thread, on cache miss event, query the GID table. Even in
> this approach, if multiple processes are used, a GID cache needs to be
> maintained on a per-process basis. With a large number of processes,
> this method doesn't scale.
>
> Hence, we introduce this series of patches, which introduces an API to
> query the complete GID tables of an RDMA device, that returns all valid
> GID table entries.
>
> This is done through single ioctl, eliminating 2304 read, 2304 open and
> 2304 close system calls to just a total of 2 calls (one for each device).
>
> While at it, we also introduce an API to query an individual GID entry
> over ioctl interface, which provides all GID attributes information.
>
> Thanks
>
> Avihai Horon (4):
> RDMA/core: Change rdma_get_gid_attr returned error code
> RDMA/core: Modify enum ib_gid_type and enum rdma_network_type
> RDMA/core: Introduce new GID table query API
> RDMA/uverbs: Expose the new GID query API to user space
I made the edit to fix the locking, please check it
Applied to for-next
Thanks,
Jason
Powered by blists - more mailing lists