Message-ID: <3866f996-6a1f-4f5a-9f92-01268a98baaf@linux.ibm.com>
Date: Wed, 22 Jan 2025 13:02:14 +0100
From: Alexandra Winter <wintera@...ux.ibm.com>
To: dust.li@...ux.alibaba.com, Julian Ruess <julianr@...ux.ibm.com>,
        Wenjia Zhang <wenjia@...ux.ibm.com>, Jan Karcher <jaka@...ux.ibm.com>,
        Gerd Bayer <gbayer@...ux.ibm.com>, Halil Pasic <pasic@...ux.ibm.com>,
        "D. Wythe" <alibuda@...ux.alibaba.com>,
        Tony Lu <tonylu@...ux.alibaba.com>, Wen Gu <guwen@...ux.alibaba.com>,
        Peter Oberparleiter
 <oberpar@...ux.ibm.com>,
        David Miller <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>, Eric Dumazet <edumazet@...gle.com>,
        Andrew Lunn <andrew+netdev@...n.ch>
Cc: Niklas Schnelle <schnelle@...ux.ibm.com>,
        Thorsten Winkler <twinkler@...ux.ibm.com>, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, Heiko Carstens <hca@...ux.ibm.com>,
        Vasily Gorbik <gor@...ux.ibm.com>,
        Alexander Gordeev
 <agordeev@...ux.ibm.com>,
        Christian Borntraeger <borntraeger@...ux.ibm.com>,
        Sven Schnelle <svens@...ux.ibm.com>, Simon Horman <horms@...nel.org>
Subject: Re: [RFC net-next 0/7] Provide an ism layer



On 22.01.25 04:04, Dust Li wrote:
> On 2025-01-20 11:28:41, Alexandra Winter wrote:
>>
>>
>> On 17.01.25 14:00, Alexandra Winter wrote:
>>>
>>>
>>> On 17.01.25 03:13, Dust Li wrote:
>>>>>>> Modular Approach: I've made the ism_loopback an independent kernel
>>>>>>> module since dynamic enable/disable functionality is not yet supported
>>>>>>> in SMC. Using insmod and rmmod for module management could provide the
>>>>>>> flexibility needed in practical scenarios.
>>>>>
>>>>> With this proposal ism_loopback is just another ism device and SMC-D will
>>>>> handle removal just like ism_client.remove(ism_dev) of other ism devices.
>>>>>
>>>>> But I understand that net/smc/ism_loopback.c today does not provide enable/disable,
>>>>> which is a big disadvantage, I agree. The ism layer is prepared for dynamic
>>>>> removal by ism_dev_unregister(). In case of this RFC that would only happen
>>>>> in case of rmmod ism. Which should be improved.
>>>>> One way to do that would be a separate ism_loopback kernel module, like you say.
>>>>> Today ism_loopback is only 10k LOC, so I'd be fine with leaving it in the ism module.
>>>>> I also think it is a great way for testing any ISM client, so it has benefit for
>>>>> anybody using the ism module.
>>>>> Another way would be e.g. an 'enable' entry in the sysfs of the loopback device.
>>>>> (Once we agree if and how to represent ism devices in genera in sysfs).
>>>> This works for me as well. I think it would be better to implement this
>>>> within the common ISM layer, rather than duplicating the code in each
>>>> device. Similar to how it's done in netdevice.
>>>>
>>>> Best regards,
>>>> Dust
>>>
>>>
>>> Is there a specific example for enable/disable in the netdevice code, you have in mind?
>>> Or do you mean in general how netdevice provides a common layer?
>>> Yes, everything that is common for all devices should be provided by the network layer.
>>
>>
>> Dust for some reason, you did not 'Reply-all':
> 
> Oh, sorry I didn't notice that
> 
>> Dust Li wrote:
>>> I think dev_close()/dev_open() are the high-level APIs, while
>>> ndo_stop()/ndo_open() are the underlying device operations that we
>>> can reference.
>>
>>
>> I hear you, it can be beneficial to have a way for upper layers to
>> enable/disable an ism device.
>> But all this is typically a tricky area. The device driver can also have
>> reasons to enable/disable a device, then hardware could do that or even
>> hotplug a device. Error recovery on different levels may want to run a
>> disable/enable sequence as a reset, etc. And all this has potential for
>> deadlocks.
>> All this is rather trivial for ism-loopback, as there is not much of a
>> lower layer.
>> ism-vpci already has 'HW' / device driver configure on/off and device
>> add/remove.
>> For a future ism-virtio, the Hipervisor may want to add/remove devices.
>>
>> I wonder what could be the simplest definition of an enable/disable for
>> the ism layer, that we can start with? More sophisticated functionality
>> can always be added later.
>> Maybe support for add/remove ism-device by the device driver is
>> sufficient as  starting point?
> 
> I agree; this can be added later. For now, we can simply support
> unregistering a device from the device driver. Which is already handled
> by ism_dev_unregister() IIUC.
> 
> However, I believe we still need an API and the ability to enable or
> disable ISM devices from the upper layer. For example, if we want to
> disable a specific ISM device (such as the loopback device) in SMC, we
> should not do so by disabling the loopback device at the device layer,
> as it may also serve other clients beyond SMC.


Just a thought: not all clients have to use all available ism devices.
The client could opt out without removing the device.

> 
> Further more, I think removing the loopback from the loopback device
> driver seems unnecessory ? Since we should support that from the upper
> layer in the future.
> 
> Best regards,
> Dust


All good points. But it also shows that there are many options how to
extend ism device handling of the upper layers / clients.
e.g. I can image a loop macro ism_for_each_dev() might be nice...
I'd prefer to take one step at a time. Start with a minimal useful ism
layer and extend by usecase.

Powered by blists - more mailing lists