netdev - Re: [PATCH v2 net-next 0/2] net: allow virtual netdevs to forward UDP L4 and fraglist GSO skbs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20201103172646.30dc9502@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Tue, 3 Nov 2020 17:26:46 -0800
From:   Jakub Kicinski <kuba@...nel.org>
To:     Alexander Lobakin <alobakin@...me>
Cc:     "David S. Miller" <davem@...emloft.net>,
        Jay Vosburgh <j.vosburgh@...il.com>,
        Veaceslav Falico <vfalico@...il.com>,
        Andy Gospodarek <andy@...yhouse.net>,
        Jiri Pirko <jiri@...nulli.us>,
        Steffen Klassert <steffen.klassert@...unet.com>,
        Willem de Bruijn <willemb@...gle.com>,
        Miaohe Lin <linmiaohe@...wei.com>,
        Antoine Tenart <atenart@...nel.org>,
        Mauro Carvalho Chehab <mchehab+huawei@...nel.org>,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2 net-next 0/2] net: allow virtual netdevs to forward
 UDP L4 and fraglist GSO skbs

On Sun, 01 Nov 2020 13:16:32 +0000 Alexander Lobakin wrote:
> NETIF_F_GSO_UDP_L4 and NETIF_F_GSO_FRAGLIST allow drivers to offload
> GSO UDP L4. This works well on simple setups, but when any logical
> netdev (e.g. VLAN) is present, kernel stack always performs software
> resegmentation which actually kills the performance.
> 
> The full path in such cases is like:
> 1. Our NIC driver advertises a support for fraglists, GSO UDP L4, GSO
>    fraglists.
> 2. User enables fraglisted GRO via Ethtool.
> 3. GRO subsystem receives UDP frames from driver and merges the packets
>    into fraglisted GSO skb(s).
> 4. Networking stack queues it up for xmitting.
> 5. Virtual device like VLAN doesn't advertise a support for GSO UDP L4
>    and GSO fraglists, so skb_gso_check() doesn't allow to pass this skb
>    as is to the real driver.
> 6. Kernel then has to form a bunch of regular UDP skbs from that one and
>    pass it to the driver instead. This fallback is *extremely* slow for
>    any GSO types, but especially for GSO fraglists.
> 7. All further processing performs with a series of plain UDP skbs, and
>    the driver gets it one-by-one, despite that it supports UDP L4 and
>    fraglisted GSO.
> 
> That's not OK because:
> a) logical/virtual netdevs like VLANs, bridges etc. should pass GSO skbs
>    as is;
> b) even if the final driver doesn't support such type of GSO, this
>    software resegmenting should be performed right before it, not in the
>    middle of processing -- I think I even saw that note somewhere in
>    kernel documentation, and it's totally reasonable in terms of
>    performance.
> 
> Despite the fact that no mainline drivers currently supports fraglist
> GSO, this should and can be easily fixed by adding UDP L4 and fraglist
> GSO to the list of GSO types that can be passed-through the logical
> interfaces (NETIF_F_GSO_SOFTWARE). After this change, no resegmentation
> occurs (if a particular driver supports and advertises this), and the
> performance goes on par with e.g. 1:1 forwarding.
> The only logical netdevs that seem to be unaffected to this are bridge
> interfaces, as their code uses full NETIF_F_GSO_MASK.
> 
> Tested on MIPS32 R2 router board with a WIP NIC driver in VLAN NAT:
> 20 Mbps baseline, 1 Gbps / link speed with this patch.
> 
> Since v1 [1]:
>  - handle bonding and team drivers as suggested by Willem de Bruijn;
>  - reword and expand the introduction with the particular example. 
> 
> [1] https://lore.kernel.org/netdev/Mx3BWGop6fGORN6Cpo4mHIHz2b1bb0eLxeMG8vsijnk@cp3-web-020.plabs.ch


Applied, thanks!