Message-ID: <Y1totRGH3t9u3QlQ@TonyMac-Alibaba>
Date:   Fri, 28 Oct 2022 13:29:25 +0800
From:   Tony Lu <tonylu@...ux.alibaba.com>
To:     Jan Karcher <jaka@...ux.ibm.com>
Cc:     "D. Wythe" <alibuda@...ux.alibaba.com>, kuba@...nel.org,
        davem@...emloft.net, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, linux-rdma@...r.kernel.org
Subject: Re: [PATCH net-next v3 00/10] optimize the parallelism of SMC-R
 connections

On Wed, Oct 26, 2022 at 03:12:48PM +0200, Jan Karcher wrote:
> 
> 
> On 25/10/2022 08:13, Tony Lu wrote:
> > On Mon, Oct 24, 2022 at 03:10:54PM +0200, Jan Karcher wrote:
> > > Hi D. Wythe,
> > > 
> > > I reply with the feedback on your fix to your v4 fix.
> > > 
> > > Regarding your questions:
> > > We are aware of this situation and we are currently evaluating how we want
> > > to deal with SMC-D in the future because as of right now i can understand
> > > your frustration regarding the SMC-D testing.
> > > Please give me some time to hit up the right people and collect some
> > > information to answer your question. I'll let you know as soon as i have an
> > > answer.
> 
> Hi Tony (and D.),
> > 
> > Hi Jan,
> > 
> > We sent a RFC [1] to mock SMC-D device for inter-VM communication. The
> > original purpose is not to test, but for now it could be useful for the
> > people who are going to test without physical devices in the community.
> 
> I'm aware of the RFC and various people in IBM looked over it.
> 
> As stated in the last mail we are aware that the entanglement between SMC-D
> and ISM is causing problems for the community.
> To give you a little insight:
> 
> In order to improve the code quality and usability for the broader community
> we are working on placing an API between SMC-D and the ISM device. If this
> API is complete it will be easier to use different "devices" for SMC-D. One
> could be your device driver for inter-VM communication (ivshmem).
> Another one could be a "Dummy-Device" which just implements the required
> interface which acts as a loopback device. This would work only in a single
> Linux instance, thus would be the perfect device to test SMC-D logic for the
> broad community.

That sounds great :-) It will provide many possibilities.

> We would hope that these changes remove the hardware restrictions and that
> the community picks up the idea and implements devices and improves SMC
> (including SMC-D and SMC-R) even more in the future!
> 
> As i said - and also teased by Alexandra in a respond to your RFC - this API
> feature is currently being developed and in our internal reviews. This would
> make your idea with the inter-VM communication a lot easier and would
> provide a clean base to build upon in the future.

Great +1.

> 
> > 
> > This driver basically works but I would improve it for testing. Before
> > that, what do you think about it?
> 
> I think it is a great idea and we should definetly give it a shot! I'm also
> putting a lot in code quality and future maintainability. The API is a key
> feature there improving the usability for the community and our work as
> maintainers. So - for the sake of the future of the SMC code base - I'd like
> to wait with putting your changes upstream for the API and use your idea to
> see if fits our (and your) requirements.

Sure. We are very much looking forward to the new API :-) Maybe we can
discuss this API in the mail list, and I'd like to adapt it first.

> 
> > 
> > And where to put this driver? In kernel with SMC code or merge into
> > separate SMC test cases. I haven't made up my mind yet.
> 
> We are not sure either currently, and have to think about that for a bit. I
> think your driver could be a classic driver, since it is usable for a real
> world problem (communication between two VMs on the same host). If we look
> at the "Dummy-Device" above we see that it does not provide any value beside
> testing. Feel free to share your ideas on that topic.

I agree with this. SMC would provides a common ability that drivers can
be introduced as classic driver for every individual device, maybe likes
ethernet cards with their drivers.

And for dummy devices, I have an idea that provides the same
shared-memory ability for same host, just like loopback devices for
TCP/IP. SMC-D with loopback devices also shows better performance compared
with other protocols.

Maybe SMC can cover all the scenes including inter-host (SMC-R),
inter-VM (SMC-D) and inter-local-process (SMC-D) communication with the
fastest path, and make things easier for user-space. I'd like to share
this RFC later when it's ready.

> 
> > 
> > [1] https://lore.kernel.org/netdev/20220720170048.20806-1-tonylu@linux.alibaba.com/
> > 
> > Cheers,
> > Tony Lu
> 
> A friendly disclaimer: Even tho this API feature is pretty far in the
> development process it can always be that we decide to drop it, if it does
> not meet our quality expectations. But of course we'll keep you updated.
> 

No problem. If there has something that we can involve, we'd be pleasure
to do that.

Cheers,
Tony Lu


> - Jan

Powered by blists - more mailing lists