linux-kernel - Re: [RFC PATCH net-next] net/smc: Introduce receive queue flow control support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6f1b9813-483b-ea2e-f6da-c349edd34003@linux.alibaba.com>
Date:   Sat, 22 Jan 2022 00:21:45 +0800
From:   Guangguan Wang <guangguan.wang@...ux.alibaba.com>
To:     dust.li@...ux.alibaba.com, kgraul@...ux.ibm.com,
        davem@...emloft.net, kuba@...nel.org
Cc:     linux-s390@...r.kernel.org, netdev@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH net-next] net/smc: Introduce receive queue flow
 control support

On 2022/1/20 17:51, dust.li wrote:
> On Thu, Jan 20, 2022 at 02:51:40PM +0800, Guangguan Wang wrote:
>> This implement rq flow control in smc-r link layer. QPs
>> communicating without rq flow control, in the previous
>> version, may result in RNR (reveive not ready) error, which
>> means when sq sends a message to the remote qp, but the
>> remote qp's rq has no valid rq entities to receive the message.
>> In RNR condition, the rdma transport layer may retransmit
>> the messages again and again until the rq has any entities,
>> which may lower the performance, especially in heavy traffic.
>> Using credits to do rq flow control can avoid the occurrence
>> of RNR.
> 
> I'm wondering if SRQ can be used to solve this problem ?
> 
> One of my concern on credit-base flow control is if the RTT is
> a bit longer, we may have to wait RTT/2 for peer to grant us credit
> before we can really send more data. That may decrease the maximium
> bandwidth we can achive in this case.

Longer RTT can result in more inflight messages and increase
the announcement latency indeed.

The following items are used in this patch to reduce the pact
of this situation.
- More rqe. (average 2 credits per smc_connection now, longer RTT is
  a good case for me to check whether an average of 2 is enough. As
  each additional rqe only increases the memory by 104 Bytes,
  SRQ may be an icing on the cake option to reduce memory usage)
- Announce frequenly. (credits carried by every cdc msg)
- Avoid credit accumulation. (announce as soon as the low watermark(1/3 rq entities) is reached)