netdev - Re: Need of advice for XDP sockets on top of the interfaces behind a Linux bonding device

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJEV1ig2Gyqb2MPHU+qN_G7XDVNZffU5HDm5VkoGev_QOe7bXg@mail.gmail.com>
Date: Tue, 30 Jan 2024 16:40:56 +0200
From: Pavel Vazharov <pavel@...e.net>
To: Toke Høiland-Jørgensen <toke@...nel.org>
Cc: Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org, 
	Magnus Karlsson <magnus.karlsson@...il.com>
Subject: Re: Need of advice for XDP sockets on top of the interfaces behind a
 Linux bonding device

On Tue, Jan 30, 2024 at 4:32 PM Toke Høiland-Jørgensen <toke@...nel.org> wrote:
>
> Pavel Vazharov <pavel@...e.net> writes:
>
> >> On Sat, Jan 27, 2024 at 7:08 AM Pavel Vazharov <pavel@...e.net> wrote:
> >>>
> >>> On Sat, Jan 27, 2024 at 6:39 AM Jakub Kicinski <kuba@...nel.org> wrote:
> >>> >
> >>> > On Sat, 27 Jan 2024 05:58:55 +0200 Pavel Vazharov wrote:
> >>> > > > Well, it will be up to your application to ensure that it is not. The
> >>> > > > XDP program will run before the stack sees the LACP management traffic,
> >>> > > > so you will have to take some measure to ensure that any such management
> >>> > > > traffic gets routed to the stack instead of to the DPDK application. My
> >>> > > > immediate guess would be that this is the cause of those warnings?
> >>> > >
> >>> > > Thank you for the response.
> >>> > > I already checked the XDP program.
> >>> > > It redirects particular pools of IPv4 (TCP or UDP) traffic to the application.
> >>> > > Everything else is passed to the Linux kernel.
> >>> > > However, I'll check it again. Just to be sure.
> >>> >
> >>> > What device driver are you using, if you don't mind sharing?
> >>> > The pass thru code path may be much less well tested in AF_XDP
> >>> > drivers.
> >>> These are the kernel version and the drivers for the 3 ports in the
> >>> above bonding.
> >>> ~# uname -a
> >>> Linux 6.3.2 #1 SMP Wed May 17 08:17:50 UTC 2023 x86_64 GNU/Linux
> >>> ~# lspci -v | grep -A 16 -e 1b:00.0 -e 3b:00.0 -e 5e:00.0
> >>> 1b:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
> >>> SFI/SFP+ Network Connection (rev 01)
> >>>        ...
> >>>         Kernel driver in use: ixgbe
> >>> --
> >>> 3b:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
> >>> SFI/SFP+ Network Connection (rev 01)
> >>>         ...
> >>>         Kernel driver in use: ixgbe
> >>> --
> >>> 5e:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
> >>> SFI/SFP+ Network Connection (rev 01)
> >>>         ...
> >>>         Kernel driver in use: ixgbe
> >>>
> >>> I think they should be well supported, right?
> >>> So far, it seems that the present usage scenario should work and the
> >>> problem is somewhere in my code.
> >>> I'll double check it again and try to simplify everything in order to
> >>> pinpoint the problem.
> > I've managed to pinpoint that forcing the copying of the packets
> > between the kernel and the user space
> > (XDP_COPY) fixes the issue with the malformed LACPDUs and the not
> > working bonding.
>
> (+Magnus)
>
> Right, okay, that seems to suggest a bug in the internal kernel copying
> that happens on XDP_PASS in zero-copy mode. Which would be a driver bug;
> any chance you could test with a different driver and see if the same
> issue appears there?
>
> -Toke
No, sorry.
We have only servers with Intel 82599ES with ixgbe drivers.
And one lab machine with Intel 82540EM with igb driver but we can't
set up bonding there
and the problem is not reproducible there.