Message-ID: <1e96806f-0a4e-4292-9483-928b1913d311@linux.ibm.com>
Date: Mon, 10 Feb 2025 10:38:27 +0100
From: Alexandra Winter <wintera@...ux.ibm.com>
To: dust.li@...ux.alibaba.com, Niklas Schnelle <schnelle@...ux.ibm.com>,
        Julian Ruess <julianr@...ux.ibm.com>,
        Wenjia Zhang <wenjia@...ux.ibm.com>, Jan Karcher <jaka@...ux.ibm.com>,
        Gerd Bayer <gbayer@...ux.ibm.com>, Halil Pasic <pasic@...ux.ibm.com>,
        "D. Wythe" <alibuda@...ux.alibaba.com>,
        Tony Lu <tonylu@...ux.alibaba.com>, Wen Gu <guwen@...ux.alibaba.com>,
        Peter Oberparleiter <oberpar@...ux.ibm.com>,
        David Miller <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>, Eric Dumazet <edumazet@...gle.com>,
        Andrew Lunn <andrew+netdev@...n.ch>
Cc: Thorsten Winkler <twinkler@...ux.ibm.com>, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, Heiko Carstens <hca@...ux.ibm.com>,
        Vasily Gorbik <gor@...ux.ibm.com>,
        Alexander Gordeev
 <agordeev@...ux.ibm.com>,
        Christian Borntraeger <borntraeger@...ux.ibm.com>,
        Sven Schnelle <svens@...ux.ibm.com>, Simon Horman <horms@...nel.org>
Subject: Re: [RFC net-next 0/7] Provide an ism layer



On 10.02.25 06:08, Dust Li wrote:
> On 2025-01-28 17:04:53, Alexandra Winter wrote:
>>
>>
>> On 18.01.25 16:31, Dust Li wrote:
>>> On 2025-01-17 11:38:39, Niklas Schnelle wrote:
>>>> On Fri, 2025-01-17 at 10:13 +0800, Dust Li wrote:
>>>>>>
>>>> ---8<---
>>>>>> Here are some of my thoughts on the matter:
>>>>>>>>
>>>>>>>> Naming and Structure: I suggest we refer to it as SHD (Shared Memory
>>>>>>>> Device) instead of ISM (Internal Shared Memory). 
>>>>>>
>>>>>>
>>>>>> So where does the 'H' come from? If you want to call it Shared Memory _D_evice?
>>>>>
>>>>> Oh, I was trying to refer to SHM(Share memory file in the userspace, see man
>>>>> shm_open(3)). SMD is also OK.
>>>>>
>>>>>>
>>>>>>
>>>>>> To my knowledge, a
>>>>>>>> "Shared Memory Device" better encapsulates the functionality we're
>>>>>>>> aiming to implement. 
>>>>>>
>>>>>>
>>>>>> Could you explain why that would be better?
>>>>>> 'Internal Shared Memory' is supposed to be a bit of a counterpart to the
>>>>>> Remote 'R' in RoCE. Not the greatest name, but it is used already by our ISM
>>>>>> devices and by ism_loopback. So what is the benefit in changing it?
>>>>>
>>>>> I believe that if we are going to separate and refine the code, and add
>>>>> a common subsystem, we should choose the most appropriate name.
>>>>>
>>>>> In my opinion, "ISM" doesn’t quite capture what the device provides.
>>>>> Since we’re adding a "Device" that enables different entities (such as
>>>>> processes or VMs) to perform shared memory communication, I think a more
>>>>> fitting name would be better. If you have any alternative suggestions,
>>>>> I’m open to them.
>>>>
>>>> I kept thinking about this a bit and I'd like to propose yet another
>>>> name for this group of devices: Memory Communication Devices (MCD)
>>>>
>>>> One important point I see is that there is a bit of a misnomer in the
>>>> existing ISM name in that our ISM device does in fact *not* share
>>>> memory in the common sense of the "shared memory" wording. Instead it
>>>> copies data between partitions of memory that share a common
>>>> cache/memory hierarchy while not sharing the memory itself. loopback-
>>>> ism and a possibly future virtio-ism on the other hand would share
>>>> memory in the "shared memory" sense. Though I'd very much hope they
>>>> will retain a copy mode to allow use in partition scenarios.
>>>>
>>>> With that background I think the common denominator between them and
>>>> the main idea behind ISM is that they facilitate communication via
>>>> memory buffers and very simple and reliable copy/share operations. I
>>>> think this would also capture our planned use-case of devices (TTYs,
>>>> block devices, framebuffers + HID etc) provided by a peer on top of
>>>> such a memory communication device.
>>>
>>> Make sense, I agree with MCD.
>>>
>>> Best regard,
>>> Dust
>>>
>>
>>
> 
> Hi Winter,
> 
> Sorry for the late reply; we were on break for the Chinese Spring
> Festival.
> 
>>
>> In the discussion with Andrew Lunn, it showed that
>> a) we need an abstract description of 'ISM' devices (noted)
>> b) DMBs (Direct Memory Buffers) are a critical differentiator.
>>
>> So what do your think of Direct Memory Communication (DMC) as class name for these devices?
>>
>> I don't have a strong preference (we could also stay with ISM). But DMC may be a bit more
>> concrete than MCD or ISM.
> 
> I personally prefer MCD over Direct Memory Communication (DMC).
> 
> For loopback or Virtio-ISM, DMC seems like a good choice. However, for
> IBM ISM, since there's a DMA copy involved, it doesn’t seem truly "Direct,"
> does it?
> 
> Additionally, since we are providing a device, MCD feels like a more
> fitting choice, as it aligns better with the concept of a "device."
> 
> Best regards,
> Dust

Thank you for your thoughts, Dust.
For me the 'D as 'direct' is not so much about the number of copies, but more about the
aspect, that you can directly write at any offset into the buffer. I.e. no queues.
More like the D in DMA or RDMA.

I am preparing a talk for netdev in March about this subject, and the more I work on it,
it seems to me that the buffers ('B'), that are
a) only authorized for a single remote device and
b) can be accessed at any offset
are the important differentiator compared other virtual devices.
So maybe 'D' for Dedicated?

I even came up with
dibs - Dedicated Internal Buffer Sharing or
dibc - Dedicated Internal Buffer Communication
(ok, I like the sound and look of the 'I'. But being on the same hardware as opposed
to RDMA is also an important aspect.)


MCD - 'memory communication device' sounds rather vague to me. But if it is the
smallest common denominator, i.e. the only thing we can all agree on, I could live with it.

Powered by blists - more mailing lists