Message-ID: <Y+RAAowqXn1JmMY4@x130>
Date:   Wed, 8 Feb 2023 16:36:18 -0800
From:   Saeed Mahameed <saeed@...nel.org>
To:     Jakub Kicinski <kuba@...nel.org>
Cc:     Jason Gunthorpe <jgg@...dia.com>,
        Leon Romanovsky <leon@...nel.org>,
        "David S. Miller" <davem@...emloft.net>,
        Paolo Abeni <pabeni@...hat.com>,
        Eric Dumazet <edumazet@...gle.com>,
        Saeed Mahameed <saeedm@...dia.com>, linux-rdma@...r.kernel.org,
        netdev@...r.kernel.org
Subject: Re: pull-request: mlx5-next 2023-01-24 V2

On 08 Feb 15:19, Jakub Kicinski wrote:
>On Wed, 8 Feb 2023 12:13:00 -0400 Jason Gunthorpe wrote:
>> On Tue, Feb 07, 2023 at 02:03:30PM -0800, Jakub Kicinski wrote:
>> > > I also would like to not discuss this :)
>> >
>> > Well, then... Suggest a delineation or a way forward if you don't like
>> > mine. The circular conversation + RDMA gets its way has to end sooner
>> > or later.
>>
>> I can't accept yours because it means RDMA stops existing. So we must
>> continue with what has been done for the last 15 years - RDMA
>> (selectively) mirrors the IP and everything running at or below the IP
>> header level.
>
>Re-implement bits you need for configuration, not stop existing.
>

Why ?? we will end up with the same code in this PULL plus some redundant
rdma API, please see explanation below.

>> > > An open source kernel implementation of a private standard for HW that
>> > > only one company can purchase that is only usable with a proprietary
>> > > userspace. Not exactly what I'd like to see.
>> >
>> > You switched your argument 180 degrees.
>> >
>> > Fist you said:
>> >
>> >   What you posted about your goals for netdev is pretty consistent with
>> >   the typical approach from a hyperscaler purchasing department: Make it
>> >   all the same. Grind the competing vendors on price.
>> >
>> > So "Make it all the same". Now you're saying hyperscalers have their
>> > own standards.
>>
>> What do you mean? "make it all the same" can be done with private or
>> open standards?
>
>Oh. If it's someone private specs its probably irrelevant to the open
>source community?
>
>> > > Ah, I stumble across stuff from time to time - KVM and related has
>> > > some interesting things. Especially with this new confidential compute
>> > > stuff. AMD just tried to get something into their mainline iommu
>> > > driver to support their out of tree kernel, for instance.
>> > >
>> > > People try to bend the rules all the time.
>> >
>> > AMD is a vendor, tho, you said "trend of large cloud operators pushing
>> > things into the kernel". I was curious to hear the hyperscaler example
>> > 'cause I'd like to be vigilant.
>>
>> I'm looking at it from the perspective of who owns, operates and
>> monetizes the propritary close source kernel fork. It is not AMD.
>>
>> AMD/Intel/ARM provided open patches to a hyperscaler(s) for their CC
>> solutions that haven't been merged yet. The hyperscaler is the one
>> that forked Linux into closed source, integrated them and is operating
>> the closed solution.
>>
>> That the vendor pushes little parts of the hyperscaler solution to the
>> kernel & ecosystem in a trickle doesn't make the sad state of affairs
>> exclusively the vendors fault, even if their name is on the patches,
>> IMHO.
>
>Sad situation. Not my employer and not in netdev, I hope.
>I may have forgotten already what brought us down this rabbit hole...
>
>> > > The ipsec patches here have almost 0 impact on netdev because it is a
>> > > tiny steering engine configuration. I'd have more sympathy to the
>> > > argument if it was consuming a huge API surface to do this.
>> >
>> > The existence of the full IPsec offload in its entirety is questionable.
>> > We let the earlier patches in trusting that you'll deliver the
>> > forwarding support. We're calling "stop" here because when the patches
>> > from this PR were posted to the list we learned for the first time
>> > that the forwarding is perhaps less real than expected.
>>
>> ipsec offload works within netdev for non switch use cases fine. I
>> would think that alone is enough to be OK for netdev.
>>
>> I have no idea how you are jumping to some conclusion that since the
>> RDMA team made their patches it somehow has anything to do with the
>> work Leon and the netdev team will deliver in future?
>
>We shouldn't reneg what was agreed on earlier.
>
>> > > He needs to fix the bugs he created and found first :)
>> > >
>> > > As far as I'm concerned TC will stay on his list until it is done.
>> >
>> > This is what I get for trusting a vendor :/
>> >
>> > If you can't make a commitment my strong recommendation is for this code
>> > to not be accepted upstream until TC patches emerge.
>>
>> This is the strongest commitment I am allowed to make in public.
>
>As priorities shift it may never happen.
>
>> I honestly have no idea why you are so fixated on TC, or what it has
>> to do with RDMA.
>
>It's a strong justification for having full xfrm offload.
>You can't forward without full offload.

This pull has nothing to do with "full" xfrm offload, 
For RoCE to exist it has to rely on netdev attributes, such as 
IP, vlan, mac, etc .. in this series we do the same for ipsec,
we setup the steering pipeline with the proper attributes for
RoCE to function.

I don't see it will be reasonable for the rdma user to setup these
attributes twice, once via netdev API and once via rdma APIs,
this will be torture for that user, just because rdma bits are not allowed
in netdev, it's exactly that, some rdma/roce bits purely mlx5_core logic,
and it has to be in mlx5_core due to the sharing of hardware resources
between rdma and netdev.

>Anything else could theoretically be improved on the SW side.
>The VF switching offload was the winning argument in the past
>discussion.
>
>> Hasn't our netdev team done enough work on TC stuff to earn some
>> faith that we do actually care about TC as part of our portfolio?
>
>Shouldn't have brought it up in the past discussion then :|
>Being asked to implement something tangential to your goals for
>the community to accept your code is hardly unheard of.

Powered by blists - more mailing lists