[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <7fe3a213-3d2e-42d5-b44b-bbd761a01bba@linux.ibm.com>
Date: Fri, 17 Nov 2023 13:35:56 +0100
From: Wenjia Zhang <wenjia@...ux.ibm.com>
To: "D. Wythe" <alibuda@...ux.alibaba.com>, kgraul@...ux.ibm.com,
jaka@...ux.ibm.com, wintera@...ux.ibm.com, guwen@...ux.alibaba.com
Cc: kuba@...nel.org, davem@...emloft.net, netdev@...r.kernel.org,
linux-s390@...r.kernel.org, linux-rdma@...r.kernel.org,
tonylu@...ux.alibaba.com, pabeni@...hat.com, edumazet@...gle.com
Subject: Re: [PATCH net v2] net/smc: avoid data corruption caused by decline
On 17.11.23 05:59, D. Wythe wrote:
> From: "D. Wythe" <alibuda@...ux.alibaba.com>
>
> We found a data corruption issue during testing of SMC-R on Redis
> applications.
>
> The benchmark has a low probability of reporting a strange error as
> shown below.
>
> "Error: Protocol error, got "\xe2" as reply type byte"
>
> Finally, we found that the retrieved error data was as follows:
>
> 0xE2 0xD4 0xC3 0xD9 0x04 0x00 0x2C 0x20 0xA6 0x56 0x00 0x16 0x3E 0x0C
> 0xCB 0x04 0x02 0x01 0x00 0x00 0x20 0x00 0x00 0x00 0x00 0x00 0x00 0x00
> 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0xE2
>
> It is quite obvious that this is a SMC DECLINE message, which means that
> the applications received SMC protocol message.
> We found that this was caused by the following situations:
>
> client server
> proposal
> ------------->
> accept
> <-------------
> confirm
> ------------->
> wait confirm
>
> failed llc confirm
> x------
> (after 2s)timeout
> wait rsp
>
> wait decline
>
> (after 1s) timeout
> (after 2s) timeout
> decline
> -------------->
> decline
> <--------------
>
> As a result, a decline message was sent in the implementation, and this
> message was read from TCP by the already-fallback connection.
>
> This patch double the client timeout as 2x of the server value,
> With this simple change, the Decline messages should never cross or
> collide (during Confirm link timeout).
>
> This issue requires an immediate solution, since the protocol updates
> involve a more long-term solution.
>
Hi D.Wythe,
I think you understood me wrong. I mean we don't need sysctl. I like the
first version more, where you just need to add some comments in the code.
Thanks,
Wenjia
Powered by blists - more mailing lists