netdev - Re: [PATCH net-next v2 10/10] net/smc: fix application data exception

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <93eddaa5-082c-c3d2-8bc0-f6aa912c9398@linux.alibaba.com>
Date:   Fri, 16 Sep 2022 13:24:17 +0800
From:   "D. Wythe" <alibuda@...ux.alibaba.com>
To:     Wen Gu <guwen@...ux.alibaba.com>, kgraul@...ux.ibm.com,
        wenjia@...ux.ibm.com
Cc:     kuba@...nel.org, davem@...emloft.net, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, linux-rdma@...r.kernel.org
Subject: Re: [PATCH net-next v2 10/10] net/smc: fix application data exception


Hi, Wen Gu

This is indeed same issues, I will fix it in the next version.

Thanks
D. Wythe


On 9/8/22 5:37 PM, Wen Gu wrote:
> 
> 
> On 2022/8/26 17:51, D. Wythe wrote:
> 
>> From: "D. Wythe" <alibuda@...ux.alibaba.com>
>>
>> After we optimize the parallel capability of SMC-R connection
>> establishment, There is a certain probability that following
>> exceptions will occur in the wrk benchmark test:
>>
>> Running 10s test @ http://11.213.45.6:80
>>    8 threads and 64 connections
>>    Thread Stats   Avg      Stdev     Max   +/- Stdev
>>      Latency     3.72ms   13.94ms 245.33ms   94.17%
>>      Req/Sec     1.96k   713.67     5.41k    75.16%
>>    155262 requests in 10.10s, 23.10MB read
>> Non-2xx or 3xx responses: 3
>>
>> We will find that the error is HTTP 400 error, which is a serious
>> exception in our test, which means the application data was
>> corrupted.
>>
>> Consider the following scenarios:
>>
>> CPU0                            CPU1
>>
>> buf_desc->used = 0;
>>                                  cmpxchg(buf_desc->used, 0, 1)
>>                                  deal_with(buf_desc)
>>
>> memset(buf_desc->cpu_addr,0);
>>
>> This will cause the data received by a victim connection to be cleared,
>> thus triggering an HTTP 400 error in the server.
>>
>> This patch exchange the order between clear used and memset, add
>> barrier to ensure memory consistency.
>>
>> Fixes: 1c5526968e27 ("net/smc: Clear memory when release and reuse buffer")
>> Signed-off-by: D. Wythe <alibuda@...ux.alibaba.com>
>> ---
>>   net/smc/smc_core.c | 5 +++--
>>   1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
>> index 84bf84c..fdad953 100644
>> --- a/net/smc/smc_core.c
>> +++ b/net/smc/smc_core.c
>> @@ -1380,8 +1380,9 @@ static void smcr_buf_unuse(struct smc_buf_desc *buf_desc, bool is_rmb,
>>           smc_buf_free(lgr, is_rmb, buf_desc);
>>       } else {
>> -        buf_desc->used = 0;
>> -        memset(buf_desc->cpu_addr, 0, buf_desc->len);
>> +        /* memzero_explicit provides potential memory barrier semantics */
>> +        memzero_explicit(buf_desc->cpu_addr, buf_desc->len);
>> +        WRITE_ONCE(buf_desc->used, 0);
>>       }
>>   }
> 
> It seems that the same issue exists in smc_buf_unuse(), Maybe it also needs to be fixed?
> 
> 
> static void smc_buf_unuse(struct smc_connection *conn,
>                struct smc_link_group *lgr)
> {
>      if (conn->sndbuf_desc) {
>          if (!lgr->is_smcd && conn->sndbuf_desc->is_vm) {
>              smcr_buf_unuse(conn->sndbuf_desc, false, lgr);
>          } else {
>              conn->sndbuf_desc->used = 0;
>              memset(conn->sndbuf_desc->cpu_addr, 0,
>                     conn->sndbuf_desc->len);
>                          ^...................
>          }
>      }
>      if (conn->rmb_desc) {
>          if (!lgr->is_smcd) {
>              smcr_buf_unuse(conn->rmb_desc, true, lgr);
>          } else {
>              conn->rmb_desc->used = 0;
>              memset(conn->rmb_desc->cpu_addr, 0,
>                     conn->rmb_desc->len +
>                     sizeof(struct smcd_cdc_msg));
>                          ^...................
>          }
>      }
> }
> 
> Thanks,
> Wen Gu