Message-ID: <CA+mtBx-B-25-nS1pVgeCiuykdFydCiay0QCwiE75P37Q3ODRCQ@mail.gmail.com>
Date:	Tue, 16 Sep 2014 11:34:48 -0700
From:	Tom Herbert <therbert@...gle.com>
To:	Or Gerlitz <gerlitz.or@...il.com>
Cc:	Jesse Gross <jesse@...ira.com>, David Miller <davem@...emloft.net>,
	Linux Netdev List <netdev@...r.kernel.org>
Subject: Re: [PATCH v2 net-next 0/7] net: foo-over-udp (fou)

On Tue, Sep 16, 2014 at 5:44 AM, Or Gerlitz <gerlitz.or@...il.com> wrote:
> On Tue, Sep 16, 2014 at 1:44 AM, Jesse Gross <jesse@...ira.com> wrote:
>> On Mon, Sep 15, 2014 at 12:15 PM, Tom Herbert <therbert@...gle.com> wrote:
>
>>> My interpretation is that NETIF_F_GSO_UDP_TUNNEL means L3/L4
>>> encapsulation over UDP, not VXLAN.
>>> If the NIC implements things properly following the generic interface then I believe it should work
>>> with various flavors of UDP encapsulation (FOU, GUE, VXLAN, VXLAN-gpe,
>>> geneve, LISP, L2TP, nvgre, or whatever else people might dream up).
>>> This presumes that any encapsulation headers doesn't require any per
>>> segment update (so no GRE csum for instance). The stack will set up
>>> inner headers as needed, which should enough to provide to devices the
>>> offsets inner IP and TCP header which are needed for the the TSO
>>> operation (outer IP and UDP can be deduced also).
>
>
>
>> From the NICs that I am familiar with this is mostly true. The main
>> part that is missing from the current implementation is a length
>> limit: just because the hardware can skip over headers to an offset
>> doesn't mean that it can do so to an arbitrary depth. For example, in
>> the NICs that are exposing VXLAN as NETIF_F_GSO_UDP_TUNNEL we can
>> probably assume that this is limited to 8 bytes. With the Intel NICs
>> that were just announced with Geneve support, this limit has been
>> increased to 64. If we add a parameter to the driver interface to
>> expose this then it should be generic across tunnels.
>
> I'm not sure to see why the length limit became our primary concern here...
>
Like Jesse mentioned above, looks like some NICs may have assumed all
encapsulation headers are eight bytes (which allows HW to implement
everything in fixed offsets). But this length is not a universal
constant: FOU is zero length encapsulation headers, GUE or geneve is
variable. The driver should really be checking if NIC can handle the
length and if it can't perform GSO in software-- I don't think we'll
need to expose this in the features.

> Fact is that we have nice set of NICs drivers in the kernel that do advertize
> the GSO_UDP_TUNNEL feature but their HW isn't capable to segment all of:
> FOU, GUE, VXLAN, VXLAN-gpe, geneve, LISP, L2TP, nvgre, or whatever
> else people might dream up, right?
>
> So we need to fix that and let each NIC properly advertize up to the
> stack what they
> can segment in HW and what not which means that networking code would have to
> do that in SW for (say) 64KB guest TCP segment that just went through
> this encapsulation.
>
> As long as Linux didn't support any UDP encapsulation other then VXLAN
> it worked,
> but soon will too easily broken, and I vote for the fix to be part of
> the FOU series, so we have
> the kernel functional also once it applied...
>
> Even if the encapsulated headers need no update per segment (and they always
> do, e.g the IP ID field of the outer IP header) still a certain HW may
> not be able
> to do TCP segmentation under any encapsulation scheme.
>
> And in that respect, I am not sure to follow on the " If the NIC
> implements things properly
> following the generic interface" comment.
>
> Or.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists