linux-kernel - Re: [PATCH v2 3/3] IB/core: Obtain subnet

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7312F593-FF71-4EEB-9D4A-6F32A4AC7AE9@oracle.com>
Date:   Thu, 3 Jun 2021 10:36:44 +0000
From:   Haakon Bugge <haakon.bugge@...cle.com>
To:     Leon Romanovsky <leon@...nel.org>
CC:     Anand Khoje <anand.a.khoje@...cle.com>,
        OFED mailing list <linux-rdma@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "dledford@...hat.com" <dledford@...hat.com>,
        "jgg@...pe.ca" <jgg@...pe.ca>
Subject: Re: [PATCH v2 3/3] IB/core: Obtain subnet_prefix from cache in IB
 devices



> On 3 Jun 2021, at 12:16, Leon Romanovsky <leon@...nel.org> wrote:
> 
> On Thu, Jun 03, 2021 at 09:29:32AM +0000, Haakon Bugge wrote:
>> 
>> 
>>> On 3 Jun 2021, at 11:07, Leon Romanovsky <leon@...nel.org> wrote:
>>> 
>>> On Thu, Jun 03, 2021 at 12:20:24PM +0530, Anand Khoje wrote:
>>>> ib_query_port() calls device->ops.query_port() to get the port
>>>> attributes. The method of querying is device driver specific.
>>>> The same function calls device->ops.query_gid() to get the GID and
>>>> extract the subnet_prefix (gid_prefix).
>>>> 
>>>> The GID and subnet_prefix are stored in a cache. But they do not get
>>>> read from the cache if the device is an Infiniband device. The
>>>> following change takes advantage of the cached subnet_prefix.
>>>> Testing with RDBMS has shown a significant improvement in performance
>>>> with this change.
>>>> 
>>>> The function ib_cache_is_initialised() is introduced because
>>>> ib_query_port() gets called early in the stage when the cache is not
>>>> built while reading port immutable property.
>>>> 
>>>> In that case, the default GID still gets read from HCA for IB link-
>>>> layer devices.
>>>> 
>>>> Fixes: fad61ad ("IB/core: Add subnet prefix to port info")
>>>> Signed-off-by: Anand Khoje <anand.a.khoje@...cle.com>
>>>> Signed-off-by: Haakon Bugge <haakon.bugge@...cle.com>
>>>> ---
>>>> drivers/infiniband/core/cache.c  | 7 ++++++-
>>>> drivers/infiniband/core/device.c | 9 +++++++++
>>>> include/rdma/ib_cache.h          | 6 ++++++
>>>> include/rdma/ib_verbs.h          | 6 ++++++
>>>> 4 files changed, 27 insertions(+), 1 deletion(-)
>>> 
>>> Can you please help me to understand how cache is updated?
>>> 
>>> There are a lot of calls to ib_query_port() and I wonder how callers can
>>> get new GID after it was changed in already initialized cache.
>> 
>> The cache is initialized when it is created, just before the bit IB_PORT_CACHE_INITIALIZED is set in flags.
>> 
>> After commit d58c23c92548 ("IB/core: Only update PKEY and GID caches on respective events"), the GID portion of the cache is updated when a IB_EVENT_GID_CHANGE event is received.
>> 
>> Before said commit, it was updated on any event.
> 
> This part is clear to me, the missing piece is to understand what will
> happen if cache and GID are not in sync because of asynchronous nature of
> events.

The calls to ib_query_port() are asynchronous with GID change. Consider the time line:

Time    HCA     cache
t0      GIDa    GIDa
t1
t2      GIDb    GIDa
t3
t4      GIDb    GIDb
t5


Prior to this commit, if ib_query_port() was called at t1 or at t3, two different GIDs would be retrieved.

With this commit, if ib_query_port() was called at t3 or t5, two different GIDs would be retrieved.

The scenario is the same, only skewed in time.


Thxs, Håkon