netdev - Re: [RFC net-next 0/7] Provide an ism layer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <8f973fe8-04d1-4659-ac90-4b76178f6055@linux.ibm.com>
Date: Mon, 20 Jan 2025 18:25:39 +0100
From: Alexandra Winter <wintera@...ux.ibm.com>
To: Andrew Lunn <andrew@...n.ch>
Cc: dust.li@...ux.alibaba.com, Niklas Schnelle <schnelle@...ux.ibm.com>,
        Julian Ruess <julianr@...ux.ibm.com>,
        Wenjia Zhang <wenjia@...ux.ibm.com>, Jan Karcher <jaka@...ux.ibm.com>,
        Gerd Bayer <gbayer@...ux.ibm.com>, Halil Pasic <pasic@...ux.ibm.com>,
        "D. Wythe" <alibuda@...ux.alibaba.com>,
        Tony Lu <tonylu@...ux.alibaba.com>, Wen Gu <guwen@...ux.alibaba.com>,
        Peter Oberparleiter <oberpar@...ux.ibm.com>,
        David Miller <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>, Eric Dumazet <edumazet@...gle.com>,
        Andrew Lunn <andrew+netdev@...n.ch>,
        Thorsten Winkler <twinkler@...ux.ibm.com>, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, Heiko Carstens <hca@...ux.ibm.com>,
        Vasily Gorbik <gor@...ux.ibm.com>,
        Alexander Gordeev
 <agordeev@...ux.ibm.com>,
        Christian Borntraeger <borntraeger@...ux.ibm.com>,
        Sven Schnelle <svens@...ux.ibm.com>, Simon Horman <horms@...nel.org>
Subject: Re: [RFC net-next 0/7] Provide an ism layer



On 20.01.25 17:01, Andrew Lunn wrote:
>> What is central to ISM is the DMB (Direct Memory Buffer). The concept
>> that there is a DMB dedicated to one writer and one reader. It is owned
>> by the reader and only this writer can write at any offset into the DMB
>> (Fabric controlled). (Reader can technically read/write as well).
>>
>> So for the client API I think the core functions are
>> - move_data(*data, target_dmb_token, offset) - called by the sending
>> client, to move data at some offset into a DMB.
> 
> Missing a length, but otherwise this looks O.K.


Right, move_data() has a length field. My bad.

> 
>> - receive_signal(dmb_token, some_signal_info) - called by the ism layer
>> to signal the client, that this DMB needs handling. (currently called
>> handle_irq)
> 
> So there is no indication where in the DMB there is new content?
> 

The existing ism implementations pass a bit mask in 'some_signal_info'
that can be used to signal which parts of the DMB have data to look at.


> And when you say "This DMB" does that imply there are multiple DMB
> shared between two peers?
> 

Yes, there can be multiple DMBs between the same two peers. And/or an
ism device can provide multiple DMBs that are shared with different peers.


> Maybe i have the wrong idea about a DMB. I was thinking of maybe 64K
> to a few Mega bytes of memory, in a memory which could truly be shared
> by CPUs. But maybe a DMB is just a 4K Page, and you have lots of them?
> If you are 'faking' a shared memory with DMA, they can be anywhere in
> the address space where the DMA engine can access them.
> 

More the latter. Although they can be large, if the client or
application wants to spend so much memory.

Which brings us back to the other thread, that ISM may not be the best
name for this concept. MCD - 'Memory Communication Device', was a
proposals without 'Shared' in the name...


>> I would not want to abstract that to a message based API, because then
>> we need queues etc and are almost at a net_device. All that is not
>> needed for ism, because DMBs are dedicated to a single writer (who has
>> the responsibility).
> 
> But i assume there are "protocols" above this. You talked about
> running a TTY over this. That should be standardized, so everybody
> implements TTYs in exactly the same way. 
> 

Yes, the 'clients' are the protocols above this.


>>> One thing we cannot hide, however, is whether the operation is zero-copy
>>> or copy. This distinction is important because we can reuse the data at
>>> different times in copy mode and zero-copy mode.
> 
> This needs more explanation. Are you talking about putting data into
> the DMB, or moving the DMB to the peer?
> 

The former: putting data into the DMB.
But yes the concept of attached, no-copy DMBs, that was introduced by
ism-loopback needs a better description.


> If you have a DMA engine
> moving stuff around, the data can be anywhere the DMA engine can
> access. But if you have a true shared memory, ideally you want to
> avoid copying into it.
> 
> Then you have the API used by your protocol drivers above. For a TTY
> running at 9600 baud, a copy into the DMB does not matter. But if you
> are talking about a network protocol stack on top, your copy from user
> space to kernel space probably wants to go direct into the DMB. So
> maybe your API also needs to include allocating/freeing DMBs in an
> abstract way so it can hide the difference between true shared memory,
> and kernel memory which can be DMAed?
> 
> 	Andrew
> 
> 
The ism_ops register_dmb() and unregister_dmb() are meant to provide
that API.