[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <e85fe903-a025-a693-906b-834ff2a2a812@linux.ibm.com>
Date: Thu, 28 Sep 2023 05:16:10 +0200
From: Jan Karcher <jaka@...ux.ibm.com>
To: Alexandra Winter <wintera@...ux.ibm.com>,
dust.li@...ux.alibaba.com, Wen Gu <guwen@...ux.alibaba.com>,
kgraul@...ux.ibm.com, wenjia@...ux.ibm.com, davem@...emloft.net,
edumazet@...gle.com, kuba@...nel.org, pabeni@...hat.com
Cc: schnelle@...ux.ibm.com, gbayer@...ux.ibm.com, pasic@...ux.ibm.com,
alibuda@...ux.alibaba.com, tonylu@...ux.alibaba.com,
linux-s390@...r.kernel.org, netdev@...r.kernel.org,
linux-kernel@...r.kernel.org
Subject: Re: [PATCH net-next v4 09/18] net/smc: introduce SMC-D loopback
device
On 26/09/2023 09:24, Alexandra Winter wrote:
>
>
> On 25.09.23 17:18, Dust Li wrote:
>>> Hello Wen Gu,
>>>
>>> thank you for adding the Kconfig, so the distributions can decide when to offer this feature.
>>>
>>> I propose you add some kind of runtime switch as well. Not every user who loads the SMC module
>>> may want to exploit smcd-loopback. Especially in native environements without containers.
>>>
>>> If no RoCE interfaces or no ISM interfaces exist, the respective handling is skipped in SMC.
>>> If loopback is always created unconditionally, there is no way to opt-out.
>> Hi Sandy,
>>
>> After talking to Wen Gu offline, I think the real issue here might be
>> we don't have an abstract layer in SMC, something like net/core/dev.c
>>
>> Without this, we cannot do:
>>
>> 1. Enable/disable those devices dynamically
>> Currently, If we want to disable a SMC-R device to communicate with
>> others, we need to refer to 'ip link set dev xxx down' to disable the
>> netdevice, then Infiniband subsystem will notify SMC that the state of
>> the IB device has changed. We cannot explicitly choose not to use some
>> specific IB/RoCE devices without disable totally.
>> If the loopback device need to support enable/disable itself, I
>> think it might be better to enable this feature for all SMC devices.
>>
>> 2. Do statistics per device
>> Now, we have to relay on IB/RoCE devices' hardware statistics to see
>> how many packets/bytes we have sent through this device.
>>
>> Both the above issues get worse when the IB/RoCE device is shared by SMC
>> and userspace RDMA applications. If SMC-R and userspace RDMA applications
>> run at the same time, we can't enable the device to run userspace RDMA
>> applications while block it from running SMC. For statistics, we cannot
>> tell how many packets/bytes were sent by SMC and how many were sent by
>> userspace RDMA applications.
>>
>> So I think those are better to support in the SMC layer.
>>
>> Best regards!
>> Dust
>
> Thank you very much for your considerations. I also think a generic handling
> of these requirements in the smc layer would be best. Especially, if we want
> to add virtio-ism support soon. There we will face the same issues again.
> Let's hear what others think about this.
>
>
Thanks you Sandy for bringing it up and Dust Li & Wen Gu for your thoughts.
I agree that such a runtime switch is needed and also that this generic
handling would be good in the smc layer.
Powered by blists - more mailing lists