Date:   Wed, 6 Oct 2021 17:27:31 +0800
From:   Hao Sun <sunhao.th@...il.com>
To:     Jamal Hadi Salim <jhs@...atatu.com>, xiyou.wangcong@...il.com,
        Jiří Pírko <jiri@...nulli.us>,
        davem@...emloft.net, kuba@...nel.org, netdev@...r.kernel.org,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: possible deadlock in sch_direct_xmit

Hello,

When using Healer to fuzz the latest Linux kernel, the following crash
was triggered.

HEAD commit: 42d43c92fc57  Merge branch 'for-linus'
git tree: upstream
console output:
https://drive.google.com/file/d/16oeU35nvvz-aFIrGtltd6N0HHf6e6LXl/view?usp=sharing
kernel config: https://drive.google.com/file/d/15vWoQRbJuuMu4ovWhUm1h4SrHyNwK8im/view?usp=sharing

Sorry, I don't have a reproducer for this crash, hope the symbolized
report can help.
If you fix this issue, please add the following tag to the commit:
Reported-by: Hao Sun <sunhao.th@...il.com>

============================================
WARNING: possible recursive locking detected
5.15.0-rc3+ #1 Not tainted
--------------------------------------------
syz-executor/9686 is trying to acquire lock:
ffff888013986498 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock
./include/linux/spinlock.h:363 [inline]
ffff888013986498 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock
./include/linux/netdevice.h:4405 [inline]
ffff888013986498 (_xmit_ETHER#2){+.-.}-{2:2}, at:
sch_direct_xmit+0x30f/0xc60 net/sched/sch_generic.c:340

but task is already holding lock:
ffff888013984898 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock
./include/linux/spinlock.h:363 [inline]
ffff888013984898 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock
./include/linux/netdevice.h:4405 [inline]
ffff888013984898 (_xmit_ETHER#2){+.-.}-{2:2}, at:
sch_direct_xmit+0x30f/0xc60 net/sched/sch_generic.c:340

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

7 locks held by syz-executor/9686:
 #0: ffffffff8b97e980 (rcu_read_lock_bh){....}-{1:2}, at:
lwtunnel_xmit_redirect ./include/net/lwtunnel.h:95 [inline]
 #0: ffffffff8b97e980 (rcu_read_lock_bh){....}-{1:2}, at:
ip_finish_output2+0x295/0x21e0 net/ipv4/ip_output.c:207
 #1: ffffffff8b97e980 (rcu_read_lock_bh){....}-{1:2}, at:
__dev_queue_xmit+0x1fc/0x3940 net/core/dev.c:4136
 #2: ffff888021cd5258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock
./include/linux/spinlock.h:373 [inline]
 #2: ffff888021cd5258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin
./include/net/sch_generic.h:173 [inline]
 #2: ffff888021cd5258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb
net/core/dev.c:3790 [inline]
 #2: ffff888021cd5258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x18bb/0x3940
net/core/dev.c:4170
 #3: ffff888013984898 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock
./include/linux/spinlock.h:363 [inline]
 #3: ffff888013984898 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock
./include/linux/netdevice.h:4405 [inline]
 #3: ffff888013984898 (_xmit_ETHER#2){+.-.}-{2:2}, at:
sch_direct_xmit+0x30f/0xc60 net/sched/sch_generic.c:340
 #4: ffffffff8b97e980 (rcu_read_lock_bh){....}-{1:2}, at:
lwtunnel_xmit_redirect ./include/net/lwtunnel.h:95 [inline]
 #4: ffffffff8b97e980 (rcu_read_lock_bh){....}-{1:2}, at:
ip_finish_output2+0x295/0x21e0 net/ipv4/ip_output.c:207
 #5: ffffffff8b97e980 (rcu_read_lock_bh){....}-{1:2}, at:
__dev_queue_xmit+0x1fc/0x3940 net/core/dev.c:4136
 #6: ffff88810be44258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock
./include/linux/spinlock.h:373 [inline]
 #6: ffff88810be44258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin
./include/net/sch_generic.h:173 [inline]
 #6: ffff88810be44258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb
net/core/dev.c:3790 [inline]
 #6: ffff88810be44258 (dev->qdisc_tx_busylock ?:
&qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x18bb/0x3940
net/core/dev.c:4170

stack backtrace:
CPU: 0 PID: 9686 Comm: syz-executor Not tainted 5.15.0-rc3+ #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.13.0-1ubuntu1.1 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 print_deadlock_bug kernel/locking/lockdep.c:2944 [inline]
 check_deadlock kernel/locking/lockdep.c:2987 [inline]
 validate_chain kernel/locking/lockdep.c:3776 [inline]
 __lock_acquire.cold+0x168/0x3c3 kernel/locking/lockdep.c:5015
 lock_acquire kernel/locking/lockdep.c:5625 [inline]
 lock_acquire+0x1ab/0x520 kernel/locking/lockdep.c:5590
 __raw_spin_lock ./include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 spin_lock ./include/linux/spinlock.h:363 [inline]
 __netif_tx_lock ./include/linux/netdevice.h:4405 [inline]
 sch_direct_xmit+0x30f/0xc60 net/sched/sch_generic.c:340
 __dev_xmit_skb net/core/dev.c:3803 [inline]
 __dev_queue_xmit+0x1b05/0x3940 net/core/dev.c:4170
 neigh_resolve_output net/core/neighbour.c:1492 [inline]
 neigh_resolve_output+0x52a/0x850 net/core/neighbour.c:1472
 neigh_output ./include/net/neighbour.h:510 [inline]
 ip_finish_output2+0x873/0x21e0 net/ipv4/ip_output.c:221
 __ip_finish_output net/ipv4/ip_output.c:299 [inline]
 __ip_finish_output+0x856/0x1450 net/ipv4/ip_output.c:281
 ip_finish_output+0x32/0x200 net/ipv4/ip_output.c:309
 NF_HOOK_COND ./include/linux/netfilter.h:296 [inline]
 ip_mc_output+0x268/0xec0 net/ipv4/ip_output.c:408
 dst_output ./include/net/dst.h:450 [inline]
 ip_local_out+0xaf/0x1a0 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x69d/0xa90 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0xf79/0x2af0 net/ipv4/ip_tunnel.c:810
 erspan_xmit+0x513/0x2ad0 net/ipv4/ip_gre.c:712
 __netdev_start_xmit ./include/linux/netdevice.h:4988 [inline]
 netdev_start_xmit ./include/linux/netdevice.h:5002 [inline]
 xmit_one net/core/dev.c:3576 [inline]
 dev_hard_start_xmit+0x1ff/0x950 net/core/dev.c:3592
 sch_direct_xmit+0x19f/0xc60 net/sched/sch_generic.c:342
 __dev_xmit_skb net/core/dev.c:3803 [inline]
 __dev_queue_xmit+0x1b05/0x3940 net/core/dev.c:4170
 neigh_resolve_output net/core/neighbour.c:1492 [inline]
 neigh_resolve_output+0x52a/0x850 net/core/neighbour.c:1472
 neigh_output ./include/net/neighbour.h:510 [inline]
 ip_finish_output2+0x873/0x21e0 net/ipv4/ip_output.c:221
 __ip_finish_output net/ipv4/ip_output.c:299 [inline]
 __ip_finish_output+0x856/0x1450 net/ipv4/ip_output.c:281
 ip_finish_output+0x32/0x200 net/ipv4/ip_output.c:309
 NF_HOOK_COND ./include/linux/netfilter.h:296 [inline]
 ip_mc_output+0x268/0xec0 net/ipv4/ip_output.c:408
 dst_output ./include/net/dst.h:450 [inline]
 ip_local_out+0xaf/0x1a0 net/ipv4/ip_output.c:126
 ip_send_skb+0x3e/0xe0 net/ipv4/ip_output.c:1555
 udp_send_skb.isra.0+0x6d2/0x11c0 net/ipv4/udp.c:966
 udp_sendmsg+0x1d86/0x2820 net/ipv4/udp.c:1253
 inet_sendmsg+0x99/0xe0 net/ipv4/af_inet.c:821
 sock_sendmsg_nosec net/socket.c:704 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:724
 ____sys_sendmsg+0x331/0x810 net/socket.c:2409
 ___sys_sendmsg+0x100/0x170 net/socket.c:2463
 __sys_sendmmsg+0x195/0x470 net/socket.c:2549
 __do_sys_sendmmsg net/socket.c:2578 [inline]
 __se_sys_sendmmsg net/socket.c:2575 [inline]
 __x64_sys_sendmmsg+0x99/0x100 net/socket.c:2575
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f9c5d76cc4d
Code: 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48
89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d
01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f9c5acd4c58 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 00007f9c5d8930a0 RCX: 00007f9c5d76cc4d
RDX: 000000000800001d RSI: 0000000020000100 RDI: 0000000000000003
RBP: 00007f9c5d7e5d80 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f9c5d8930a0
R13: 00007ffff72e0cbf R14: 00007ffff72e0e60 R15: 00007f9c5acd4dc0

Powered by blists - more mailing lists