Message-ID: <20250211015729.GB81943@linux.alibaba.com>
Date: Tue, 11 Feb 2025 09:57:29 +0800
From: Dust Li <dust.li@...ux.alibaba.com>
To: Alexandra Winter <wintera@...ux.ibm.com>,
	Niklas Schnelle <schnelle@...ux.ibm.com>,
	Julian Ruess <julianr@...ux.ibm.com>,
	Wenjia Zhang <wenjia@...ux.ibm.com>,
	Jan Karcher <jaka@...ux.ibm.com>, Gerd Bayer <gbayer@...ux.ibm.com>,
	Halil Pasic <pasic@...ux.ibm.com>,
	"D. Wythe" <alibuda@...ux.alibaba.com>,
	Tony Lu <tonylu@...ux.alibaba.com>,
	Wen Gu <guwen@...ux.alibaba.com>,
	Peter Oberparleiter <oberpar@...ux.ibm.com>,
	David Miller <davem@...emloft.net>,
	Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>,
	Eric Dumazet <edumazet@...gle.com>,
	Andrew Lunn <andrew+netdev@...n.ch>
Cc: Thorsten Winkler <twinkler@...ux.ibm.com>, netdev@...r.kernel.org,
	linux-s390@...r.kernel.org, Heiko Carstens <hca@...ux.ibm.com>,
	Vasily Gorbik <gor@...ux.ibm.com>,
	Alexander Gordeev <agordeev@...ux.ibm.com>,
	Christian Borntraeger <borntraeger@...ux.ibm.com>,
	Sven Schnelle <svens@...ux.ibm.com>,
	Simon Horman <horms@...nel.org>
Subject: Re: [RFC net-next 0/7] Provide an ism layer

On 2025-02-10 10:38:27, Alexandra Winter wrote:
>
>
>On 10.02.25 06:08, Dust Li wrote:
>> On 2025-01-28 17:04:53, Alexandra Winter wrote:
>>>
>>>
>>> On 18.01.25 16:31, Dust Li wrote:
>>>> On 2025-01-17 11:38:39, Niklas Schnelle wrote:
>>>>> On Fri, 2025-01-17 at 10:13 +0800, Dust Li wrote:
>>>>>>>
>>>>> ---8<---
>>>>>>> Here are some of my thoughts on the matter:
>>>>>>>>>
>>>>>>>>> Naming and Structure: I suggest we refer to it as SHD (Shared Memory
>>>>>>>>> Device) instead of ISM (Internal Shared Memory). 
>>>>>>>
>>>>>>>
>>>>>>> So where does the 'H' come from? If you want to call it Shared Memory _D_evice?
>>>>>>
>>>>>> Oh, I was trying to refer to SHM(Share memory file in the userspace, see man
>>>>>> shm_open(3)). SMD is also OK.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> To my knowledge, a
>>>>>>>>> "Shared Memory Device" better encapsulates the functionality we're
>>>>>>>>> aiming to implement. 
>>>>>>>
>>>>>>>
>>>>>>> Could you explain why that would be better?
>>>>>>> 'Internal Shared Memory' is supposed to be a bit of a counterpart to the
>>>>>>> Remote 'R' in RoCE. Not the greatest name, but it is used already by our ISM
>>>>>>> devices and by ism_loopback. So what is the benefit in changing it?
>>>>>>
>>>>>> I believe that if we are going to separate and refine the code, and add
>>>>>> a common subsystem, we should choose the most appropriate name.
>>>>>>
>>>>>> In my opinion, "ISM" doesn’t quite capture what the device provides.
>>>>>> Since we’re adding a "Device" that enables different entities (such as
>>>>>> processes or VMs) to perform shared memory communication, I think a more
>>>>>> fitting name would be better. If you have any alternative suggestions,
>>>>>> I’m open to them.
>>>>>
>>>>> I kept thinking about this a bit and I'd like to propose yet another
>>>>> name for this group of devices: Memory Communication Devices (MCD)
>>>>>
>>>>> One important point I see is that there is a bit of a misnomer in the
>>>>> existing ISM name in that our ISM device does in fact *not* share
>>>>> memory in the common sense of the "shared memory" wording. Instead it
>>>>> copies data between partitions of memory that share a common
>>>>> cache/memory hierarchy while not sharing the memory itself. loopback-
>>>>> ism and a possibly future virtio-ism on the other hand would share
>>>>> memory in the "shared memory" sense. Though I'd very much hope they
>>>>> will retain a copy mode to allow use in partition scenarios.
>>>>>
>>>>> With that background I think the common denominator between them and
>>>>> the main idea behind ISM is that they facilitate communication via
>>>>> memory buffers and very simple and reliable copy/share operations. I
>>>>> think this would also capture our planned use-case of devices (TTYs,
>>>>> block devices, framebuffers + HID etc) provided by a peer on top of
>>>>> such a memory communication device.
>>>>
>>>> Make sense, I agree with MCD.
>>>>
>>>> Best regard,
>>>> Dust
>>>>
>>>
>>>
>> 
>> Hi Winter,
>> 
>> Sorry for the late reply; we were on break for the Chinese Spring
>> Festival.
>> 
>>>
>>> In the discussion with Andrew Lunn, it showed that
>>> a) we need an abstract description of 'ISM' devices (noted)
>>> b) DMBs (Direct Memory Buffers) are a critical differentiator.
>>>
>>> So what do your think of Direct Memory Communication (DMC) as class name for these devices?
>>>
>>> I don't have a strong preference (we could also stay with ISM). But DMC may be a bit more
>>> concrete than MCD or ISM.
>> 
>> I personally prefer MCD over Direct Memory Communication (DMC).
>> 
>> For loopback or Virtio-ISM, DMC seems like a good choice. However, for
>> IBM ISM, since there's a DMA copy involved, it doesn’t seem truly "Direct,"
>> does it?
>> 
>> Additionally, since we are providing a device, MCD feels like a more
>> fitting choice, as it aligns better with the concept of a "device."
>> 
>> Best regards,
>> Dust
>
>Thank you for your thoughts, Dust.
>For me the 'D as 'direct' is not so much about the number of copies, but more about the
>aspect, that you can directly write at any offset into the buffer. I.e. no queues.
>More like the D in DMA or RDMA.

Thanks for you explaintion. I think I understand you now.

>
>I am preparing a talk for netdev in March about this subject, and the more I work on it,
>it seems to me that the buffers ('B'), that are
>a) only authorized for a single remote device and
>b) can be accessed at any offset
>are the important differentiator compared other virtual devices.
>So maybe 'D' for Dedicated?
>
>I even came up with
>dibs - Dedicated Internal Buffer Sharing or
>dibc - Dedicated Internal Buffer Communication
>(ok, I like the sound and look of the 'I'. But being on the same hardware as opposed
>to RDMA is also an important aspect.)
>
>
>MCD - 'memory communication device' sounds rather vague to me. But if it is the
>smallest common denominator, i.e. the only thing we can all agree on, I could live with it.
>

I've thought about it a bit more. Since DMC is at the same level as RDMA
and fits well with the "D" in SMC-D, either DMC or MCD works for me.

Best regards,
Dust

Powered by blists - more mailing lists