netdev - Re: new splat

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z-wYH-gIvMd89-3d@mini-arch>
Date: Tue, 1 Apr 2025 09:45:19 -0700
From: Stanislav Fomichev <stfomichev@...il.com>
To: Justin Iurman <justin.iurman@...ege.be>
Cc: Paolo Abeni <pabeni@...hat.com>,
	Alexei Starovoitov <alexei.starovoitov@...il.com>,
	bpf <bpf@...r.kernel.org>,
	Network Development <netdev@...r.kernel.org>,
	Jakub Kicinski <kuba@...nel.org>
Subject: Re: new splat

On 04/01, Justin Iurman wrote:
> On 3/31/25 10:07, Paolo Abeni wrote:
> > Adding Justin.
> > 
> > On 3/31/25 1:28 AM, Alexei Starovoitov wrote:
> > > After bpf fast forward we see this new failure:
> > > 
> > > [  138.359852] BUG: using __this_cpu_read() in preemptible [00000000]
> > > code: test_progs/9368
> > > [  138.362686] caller is lwtunnel_xmit+0x1c/0x2e0
> > > [  138.364363] CPU: 9 UID: 0 PID: 9368 Comm: test_progs Tainted: G
> > >        O        6.14.0-10767-g8be3a12f9f26 #1092 PREEMPT
> > > [  138.364366] Tainted: [O]=OOT_MODULE
> > > [  138.364366] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> > > [  138.364368] Call Trace:
> > > [  138.364370]  <TASK>
> > > [  138.364375]  dump_stack_lvl+0x80/0x90
> > > [  138.364381]  check_preemption_disabled+0xc6/0xe0
> > > [  138.364385]  lwtunnel_xmit+0x1c/0x2e0
> > > [  138.364387]  ip_finish_output2+0x2f9/0x850
> > > [  138.364391]  ? __ip_finish_output+0xa0/0x320
> > > [  138.364394]  ip_send_skb+0x3f/0x90
> > > [  138.364397]  udp_send_skb+0x1a6/0x3d0
> > > [  138.364402]  udp_sendmsg+0x87b/0x1000
> > > [  138.364404]  ? ip_frag_init+0x60/0x60
> > > [  138.364406]  ? reacquire_held_locks+0xcd/0x1f0
> > > [  138.364414]  ? copy_process+0x2ae0/0x2fa0
> > > [  138.364418]  ? inet_autobind+0x41/0x60
> > > [  138.364420]  ? __local_bh_enable_ip+0x79/0xe0
> > > [  138.364422]  ? inet_autobind+0x41/0x60
> > > [  138.364424]  ? inet_send_prepare+0xe7/0x1e0
> > > [  138.364428]  __sock_sendmsg+0x38/0x70
> > > [  138.364432]  ____sys_sendmsg+0x1c9/0x200
> > > [  138.364437]  ___sys_sendmsg+0x73/0xa0
> > > [  138.364444]  ? __fget_files+0xb9/0x180
> > > [  138.364447]  ? lock_release+0x131/0x280
> > > [  138.364450]  ? __fget_files+0xc3/0x180
> > > [  138.364453]  __sys_sendmsg+0x5a/0xa0
> > 
> > Possibly a decoded stack trace could help.
> > 
> > I think a possible suspect is:
> > 
> > commit 986ffb3a57c5650fb8bf6d59a8f0f07046abfeb6
> > Author: Justin Iurman <justin.iurman@...ege.be>
> > Date:   Fri Mar 14 13:00:46 2025 +0100
> > 
> >      net: lwtunnel: fix recursion loops
> > 
> > with dev_xmit_recursion() in lwtunnel_xmit() being called in preemptible
> > scope.
> 
> Correct, I came to the same conclusion based on that trace. However, I can't
> reproduce it with a PREEMPT kernel. It goes through without problem and the
> output is (as expected), i.e., "lwtunnel_xmit(): recursion limit reached on
> datapath".

For me adding the following to the config did the trick:
CONFIG_PREEMPT
CONFIG_DEBUG_PREEMPT

And reverting your patch made it go away.