Message-ID: <20250122141026.GO89233@linux.alibaba.com>
Date: Wed, 22 Jan 2025 22:10:26 +0800
From: Dust Li <dust.li@...ux.alibaba.com>
To: Alexandra Winter <wintera@...ux.ibm.com>,
	Julian Ruess <julianr@...ux.ibm.com>,
	Wenjia Zhang <wenjia@...ux.ibm.com>,
	Jan Karcher <jaka@...ux.ibm.com>, Gerd Bayer <gbayer@...ux.ibm.com>,
	Halil Pasic <pasic@...ux.ibm.com>,
	"D. Wythe" <alibuda@...ux.alibaba.com>,
	Tony Lu <tonylu@...ux.alibaba.com>,
	Wen Gu <guwen@...ux.alibaba.com>,
	Peter Oberparleiter <oberpar@...ux.ibm.com>,
	David Miller <davem@...emloft.net>,
	Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>,
	Eric Dumazet <edumazet@...gle.com>,
	Andrew Lunn <andrew+netdev@...n.ch>
Cc: Niklas Schnelle <schnelle@...ux.ibm.com>,
	Thorsten Winkler <twinkler@...ux.ibm.com>, netdev@...r.kernel.org,
	linux-s390@...r.kernel.org, Heiko Carstens <hca@...ux.ibm.com>,
	Vasily Gorbik <gor@...ux.ibm.com>,
	Alexander Gordeev <agordeev@...ux.ibm.com>,
	Christian Borntraeger <borntraeger@...ux.ibm.com>,
	Sven Schnelle <svens@...ux.ibm.com>,
	Simon Horman <horms@...nel.org>
Subject: Re: [RFC net-next 0/7] Provide an ism layer

On 2025-01-22 13:05:57, Alexandra Winter wrote:
>
>
>On 22.01.25 13:02, Alexandra Winter wrote:
>> 
>> 
>> On 22.01.25 04:04, Dust Li wrote:
>>> On 2025-01-20 11:28:41, Alexandra Winter wrote:
>>>>
>>>>
>>>> On 17.01.25 14:00, Alexandra Winter wrote:
>>>>>
>>>>>
>>>>> On 17.01.25 03:13, Dust Li wrote:
>>>>>>>>> Modular Approach: I've made the ism_loopback an independent kernel
>>>>>>>>> module since dynamic enable/disable functionality is not yet supported
>>>>>>>>> in SMC. Using insmod and rmmod for module management could provide the
>>>>>>>>> flexibility needed in practical scenarios.
>>>>>>>
>>>>>>> With this proposal ism_loopback is just another ism device and SMC-D will
>>>>>>> handle removal just like ism_client.remove(ism_dev) of other ism devices.
>>>>>>>
>>>>>>> But I understand that net/smc/ism_loopback.c today does not provide enable/disable,
>>>>>>> which is a big disadvantage, I agree. The ism layer is prepared for dynamic
>>>>>>> removal by ism_dev_unregister(). In case of this RFC that would only happen
>>>>>>> in case of rmmod ism. Which should be improved.
>>>>>>> One way to do that would be a separate ism_loopback kernel module, like you say.
>>>>>>> Today ism_loopback is only 10k LOC, so I'd be fine with leaving it in the ism module.
>>>>>>> I also think it is a great way for testing any ISM client, so it has benefit for
>>>>>>> anybody using the ism module.
>>>>>>> Another way would be e.g. an 'enable' entry in the sysfs of the loopback device.
>>>>>>> (Once we agree if and how to represent ism devices in genera in sysfs).
>>>>>> This works for me as well. I think it would be better to implement this
>>>>>> within the common ISM layer, rather than duplicating the code in each
>>>>>> device. Similar to how it's done in netdevice.
>>>>>>
>>>>>> Best regards,
>>>>>> Dust
>>>>>
>>>>>
>>>>> Is there a specific example for enable/disable in the netdevice code, you have in mind?
>>>>> Or do you mean in general how netdevice provides a common layer?
>>>>> Yes, everything that is common for all devices should be provided by the network layer.
>>>>
>>>>
>>>> Dust for some reason, you did not 'Reply-all':
>>>
>>> Oh, sorry I didn't notice that
>>>
>>>> Dust Li wrote:
>>>>> I think dev_close()/dev_open() are the high-level APIs, while
>>>>> ndo_stop()/ndo_open() are the underlying device operations that we
>>>>> can reference.
>>>>
>>>>
>>>> I hear you, it can be beneficial to have a way for upper layers to
>>>> enable/disable an ism device.
>>>> But all this is typically a tricky area. The device driver can also have
>>>> reasons to enable/disable a device, then hardware could do that or even
>>>> hotplug a device. Error recovery on different levels may want to run a
>>>> disable/enable sequence as a reset, etc. And all this has potential for
>>>> deadlocks.
>>>> All this is rather trivial for ism-loopback, as there is not much of a
>>>> lower layer.
>>>> ism-vpci already has 'HW' / device driver configure on/off and device
>>>> add/remove.
>>>> For a future ism-virtio, the Hipervisor may want to add/remove devices.
>>>>
>>>> I wonder what could be the simplest definition of an enable/disable for
>>>> the ism layer, that we can start with? More sophisticated functionality
>>>> can always be added later.
>>>> Maybe support for add/remove ism-device by the device driver is
>>>> sufficient as  starting point?
>>>
>>> I agree; this can be added later. For now, we can simply support
>>> unregistering a device from the device driver. Which is already handled
>>> by ism_dev_unregister() IIUC.
>>>
>>> However, I believe we still need an API and the ability to enable or
>>> disable ISM devices from the upper layer. For example, if we want to
>>> disable a specific ISM device (such as the loopback device) in SMC, we
>>> should not do so by disabling the loopback device at the device layer,
>>> as it may also serve other clients beyond SMC.
>> 
>> 
>> Just a thought: not all clients have to use all available ism devices.
>> The client could opt out without removing the device.
>> 
>>>
>>> Further more, I think removing the loopback from the loopback device
>>> driver seems unnecessory ? Since we should support that from the upper
>>> layer in the future.
>
>
>If it is not too much effort, I would like to have a simple remove for
>ism_loopback soon, as it would allow for simple variations of testcases.

Yes, this is very useful for testing before we can do that from the
upper layer.

>
>
>>>
>>> Best regards,
>>> Dust
>> 
>> 
>> All good points. But it also shows that there are many options how to
>> extend ism device handling of the upper layers / clients.
>> e.g. I can image a loop macro ism_for_each_dev() might be nice...
>> I'd prefer to take one step at a time. Start with a minimal useful ism
>> layer and extend by usecase.

That works for me.

Best regards,
Dust

Powered by blists - more mailing lists