linux-kernel - Re: [PATCH net-next] net/smc: introduce autosplit for smc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <cf07ec76-9d48-4bff-99f6-0842b5127c81@linux.ibm.com>
Date: Thu, 11 Jul 2024 17:57:59 +0200
From: Wenjia Zhang <wenjia@...ux.ibm.com>
To: Guangguan Wang <guangguan.wang@...ux.alibaba.com>, jaka@...ux.ibm.com,
        davem@...emloft.net, edumazet@...gle.com, kuba@...nel.org,
        pabeni@...hat.com
Cc: alibuda@...ux.alibaba.com, tonylu@...ux.alibaba.com,
        guwen@...ux.alibaba.com, linux-s390@...r.kernel.org,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH net-next] net/smc: introduce autosplit for smc



On 09.07.24 18:05, Guangguan Wang wrote:
> When sending large size data in TCP, the data will be split into
> several segments(packets) to transfer due to MTU config. And in
> the receive side, application can be woken up to recv data every
> packet arrived, the data transmission and data recv copy are
> pipelined.
> 
> But for SMC-R, it will transmit as many data as possible in one
> RDMA WRITE and a CDC msg follows the RDMA WRITE, in the receive
> size, the application only be woken up to recv data when all RDMA
> WRITE data and the followed CDC msg arrived. The data transmission
> and data recv copy are sequential.
> 
> This patch introduce autosplit for SMC, which can automatic split
> data into several segments and every segment transmitted by one RDMA
> WRITE when sending large size data in SMC. Because of the split, the
> data transmission and data send copy can be pipelined in the send side,
> and the data transmission and data recv copy can be pipelined in the
> receive side. Thus autosplit helps improving latency performance when
> sending large size data. The autosplit also works for SMC-D.
> 
> This patch also introduce a sysctl names autosplit_size for configure
> the max size of the split segment, whose default value is 128KiB
> (128KiB perform best in my environment).
> 
> The sockperf benchmark shows 17%-28% latency improvement when msgsize
>> = 256KB for SMC-R, 15%-32% latency improvement when msgsize >= 256KB
> for SMC-D with smc-loopback.
> 
> Test command:
> sockperf sr --tcp -m 1048575
> sockperf pp --tcp -i <server ip> -m <msgsize> -t 20
> 
> Test config:
> sysctl -w net.smc.wmem=524288
> sysctl -w net.smc.rmem=524288
> 
> Test results:
> SMC-R
> msgsize   noautosplit    autosplit
> 128KB       55.546 us     55.763 us
> 256KB       83.537 us     69.743 us (17% improve)
> 512KB      138.306 us    100.313 us (28% improve)
> 1MB        273.702 us    197.222 us (28% improve)
> 
> SMC-D with smc-loopback
> msgsize   noautosplit    autosplit
> 128KB       14.672 us     14.690 us
> 256KB       28.277 us     23.958 us (15% improve)
> 512KB       63.047 us     45.339 us (28% improve)
> 1MB        129.306 us     87.278 us (32% improve)
> 
> Signed-off-by: Guangguan Wang <guangguan.wang@...ux.alibaba.com>
> ---
>   Documentation/networking/smc-sysctl.rst | 11 +++++++++++
>   include/net/netns/smc.h                 |  1 +
>   net/smc/smc_sysctl.c                    | 12 ++++++++++++
>   net/smc/smc_tx.c                        | 19 ++++++++++++++++++-
>   4 files changed, 42 insertions(+), 1 deletion(-)
> 

Hi Guangguan,

If I remember correctly, the intention to use one RDMA-write for a 
possible large data is to reduce possible many partial stores. Since 
many year has gone, I'm not that sure if it would still be an issue. I 
need some time to check on it.

BTW, I don't really like the idea to use sysctl to set the 
autosplit_size in any value at will. That makes no sense to improve the 
performance.

Thanks,
Wenjia