netdev - Re: BUG(?): igc link up and XDP program init fails

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <723778e997db9704b130bb7d85d517ced4f6332a.camel@ericsson.com>
Date: Mon, 4 Sep 2023 12:29:35 +0000
From: Ferenc Fejes <ferenc.fejes@...csson.com>
To: "intel-wired-lan@...ts.osuosl.org" <intel-wired-lan@...ts.osuosl.org>,
	"vinicius.gomes@...el.com" <vinicius.gomes@...el.com>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"jesse.brandeburg@...el.com" <jesse.brandeburg@...el.com>,
	"sasha.neftin@...el.com" <sasha.neftin@...el.com>,
	"anthony.l.nguyen@...el.com" <anthony.l.nguyen@...el.com>
CC: "hawk@...nel.org" <hawk@...nel.org>
Subject: Re: BUG(?): igc link up and XDP program init fails

Hi Vinicius!

Sorry for the delay.

On Fri, 2023-08-25 at 15:56 -0700, Vinicius Costa Gomes wrote:
> Hi Ferenc,
> 
> Ferenc Fejes <ferenc.fejes@...csson.com> writes:
> 
> > Dear igc Maintainers!
> > 
> > I noticed that ip link set dev up fails with igc (Intel i225)
> > driver
> > when XDP program is attached to it. More precisely, only when we
> > have
> > incoming traffic and the incoming packet rate is too fast (like 100
> > packets per-sec).
> > 
> > I don't have a very smart reproducer, so 4 i225 cards are needed to
> > trigger it. My setup (enp3s0 and enp4s0 directly connected with a
> > cable, similarly enp6s0 and enp7s0).
> > 
> > veth0 ----> veth1 --redir---> enp3s0 ~~~~~~~ enp4s0
> > 			  |
> >                           +-> enp6s0 ~~~~~~~ enp7s0
> > 
> > ip link add dev type veth
> > ip nei change 1.2.3.4 lladdr aa:aa:aa:aa:aa:aa dev veth0
> > xdp-bench redirect-multi veth1 enp3s0 enp6s0	#in terminal 1
> > xdpdump -i enp4s0				#in terminal 2
> > ping -I veth0 1.2.3.4 -i 0.5 #slow packet rate  #in terminal 3
> > 
> 
> I was just able to reproduce this issue, with a different setup: 
> 
> >             System A                 |   System B   | 
> veth0 ----> veth1 --redir---> "enp3s0" ~~~~~~~ "enp4s0"
> 
> And running xdp-bench like this:
> 
> $ xdp-bench redirect-multi veth1 enp3s0
> 
> Also I am running a different traffic generator.

Thank you for the simplistic reproducer and the patch with the fix!

I confirm your patch posted in this thread solve the issue.

I tested with your reproducer, my reproducer and my original prototype
code which triggered the bug first time - all of them works. I tested
with 100k pps and still OK.


> 
> > Now in a separate terminal do a "ip link set dev enp4s0 down" and
> > "ip
> > link set dev enp4s0 up". After a while, xdpdump will see the
> > incoming
> > packets.
> > 
> 
> It seems that anything that triggers a reset of the adapter would
> trigger the bug: I am able to trigger the bug when I run 'xdp-bench'
> last (after "ping"/traffic generator), no need for 'link down/link
> up'.
> 
> > Now in terminal 3, change the ping to a faster rate:
> > ping -I veth0 1.2.3.4 -i 0.01
> > 
> > And do the ip link down/up again. In my setup, I no longer see
> > incoming
> > packets. With bpftrace I see the driver keep trying to initialize
> > itself in an endless loop.
> > 
> > Now stop the ping, wait about 4-5 seconds, and start the ping
> > again.
> > This is enough time for the driver to initialize properly, and
> > packets
> > are visible in xdpdump again.
> > 
> > If anyone has an idea what is wrong with my setup I would be happy
> > to
> > hear it, and I can help with testing fixes if this is indeed a bug.
> > I have replicated the setup with veths and it looks fine.
> > 
> 
> I don't think there's anything wrong with your setup.
> 
> I am considering this a bug, I don't have any patches from the top of
> my
> head for you to try, but taking a look.
> 
> Anyway, thanks for the report.

Feel free apply Tested-by: Ferenc Fejes <ferenc.fejes@...csson.com>
in case of submitting it to net.

> 
> 
> Cheers,

Best,
Ferenc