Message-ID: <20230726071254.GA1380402@unreal>
Date:   Wed, 26 Jul 2023 10:12:54 +0300
From:   Leon Romanovsky <leon@...nel.org>
To:     Daniel Borkmann <daniel@...earbox.net>
Cc:     syzbot <syzbot+14736e249bce46091c18@...kaller.appspotmail.com>,
        andrii@...nel.org, ast@...nel.org, bpf@...r.kernel.org,
        haoluo@...gle.com, john.fastabend@...il.com, jolsa@...nel.org,
        kpsingh@...nel.org, linux-kernel@...r.kernel.org,
        martin.lau@...ux.dev, netdev@...r.kernel.org, sdf@...gle.com,
        song@...nel.org, syzkaller-bugs@...glegroups.com, yhs@...com,
        Gal Pressman <gal@...dia.com>
Subject: Re: [syzbot] [bpf?] WARNING: ODEBUG bug in tcx_uninstall

On Fri, Jul 21, 2023 at 02:52:14AM +0200, Daniel Borkmann wrote:
> On 7/20/23 5:06 PM, syzbot wrote:
> > Hello,
> > 
> > syzbot found the following issue on:
> > 
> > HEAD commit:    03b123debcbc tcp: tcp_enter_quickack_mode() should be static
> > git tree:       net-next
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=17ac9ffaa80000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=32e3dcc11fd0d297
> > dashboard link: https://syzkaller.appspot.com/bug?extid=14736e249bce46091c18
> > compiler:       gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=133f36c6a80000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=11a8e73aa80000
> > 
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/348462fb61fa/disk-03b123de.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/33375730f77f/vmlinux-03b123de.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/b6882fbac041/bzImage-03b123de.xz
> > 
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+14736e249bce46091c18@...kaller.appspotmail.com
> 
> Thanks, I'll take a look this evening.

Did anybody post a fix for that?

We are experiencing the following kernel panic in netdev commit
b57e0d48b300 (net-next/main) Merge branch '100GbE' of git://git.kernel.org/pub/scm/linux/kernel/git/tnguy/next-queue

 [  935.131864] ------------[ cut here ]------------
 [  935.133223] WARNING: CPU: 7 PID: 32248 at kernel/bpf/tcx.c:114 tcx_uninstall+0x158/0x1a0
 [  935.135408] Modules linked in: act_tunnel_key vxlan act_mirred act_skbedit cls_matchall nfnetlink_cttimeout act_gact cls_flower sch_ingress bonding mlx5_vfio_pci vfio_pci vfio_pci_core vfio_iommu_type1 vfio mlx5_ib mlx5_core ip6_gre ib_ipoib nf_tables rdma_ucm ib_uverbs geneve ip_gre gre ip6_tunnel tunnel6 ipip tunnel4 ib_umad iptable_raw openvswitch nsh xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcrdma ib_iser libiscsi scsi_transport_iscsi rdma_cm iw_cm ib_cm rpcsec_gss_krb5 auth_rpcgss oid_registry ib_core overlay zram zsmalloc fuse [last unloaded: ib_uverbs]
 [  935.141679] CPU: 7 PID: 32248 Comm: devlink Not tainted 6.5.0-rc2_net_next_mlx5_89edf40 #1
 [  935.142577] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
 [  935.143758] RIP: 0010:tcx_uninstall+0x158/0x1a0
 [  935.144297] Code: f0 fe ff ff ba 4c 00 00 00 48 c7 c6 e7 33 27 82 48 c7 c7 c0 32 27 82 c6 05 5c 0a 3d 01 01 e8 4f a7 e8 ff 0f 0b e9 ca fe ff ff <0f> 0b eb 9d 44 0f b6 35 45 0a 3d 01 41 80 fe 01 0f 87 f6 8b 91 00
 [  935.146192] RSP: 0018:ffff8881eb853928 EFLAGS: 00010202
 [  935.146789] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000
 [  935.147548] RDX: ffff8881066f2808 RSI: ffff888106c40000 RDI: 0000000000000000
 [  935.148328] RBP: ffff8881066f2808 R08: 0000000000000001 R09: 00000000000003fe
 [  935.149081] R10: 0000000000000001 R11: 00000000fa83b2da R12: 0000000000000001
 [  935.149847] R13: ffff8881066f2808 R14: dead000000000122 R15: dead000000000100
 [  935.150490] FS:  00007fa48de38800(0000) GS:ffff88852cb80000(0000) knlGS:0000000000000000
 [  935.151213] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 [  935.151753] CR2: 00007fa48dd7b1e0 CR3: 0000000125f06002 CR4: 0000000000370ea0
 [  935.152378] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 [  935.153000] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
 [  935.153620] Call Trace:
 [  935.153918]  <TASK>
 [  935.154194]  ? __warn+0x79/0x120
 [  935.154545]  ? tcx_uninstall+0x158/0x1a0
 [  935.154936]  ? report_bug+0x17c/0x190
 [  935.155320]  ? handle_bug+0x3c/0x60
 [  935.155686]  ? exc_invalid_op+0x14/0x70
 [  935.156088]  ? asm_exc_invalid_op+0x16/0x20
 [  935.156499]  ? tcx_uninstall+0x158/0x1a0
 [  935.156896]  ? tcx_uninstall+0x57/0x1a0
 [  935.157287]  unregister_netdevice_many_notify+0x32f/0x960
 [  935.157790]  unregister_netdevice_queue+0x8d/0xe0
 [  935.158236]  unregister_netdev+0x18/0x20
 [  935.158630]  mlx5e_vport_rep_unload+0x30/0x90 [mlx5_core]
 [  935.159221]  esw_offloads_unload_rep+0x24/0x40 [mlx5_core]
 [  935.159777]  mlx5_eswitch_unload_vf_vports+0x7a/0xc0 [mlx5_core]
 [  935.160358]  mlx5_eswitch_disable_pf_vf_vports+0x15/0xa0 [mlx5_core]
 [  935.160953]  esw_offloads_disable+0xe/0x60 [mlx5_core]
 [  935.161469]  mlx5_eswitch_disable_locked+0x15a/0x180 [mlx5_core]
 [  935.162058]  mlx5_devlink_eswitch_mode_set+0xad/0x380 [mlx5_core]
 [  935.162637]  ? devlink_get_from_attrs_lock+0x9e/0x110
 [  935.163108]  devlink_nl_cmd_eswitch_set_doit+0x60/0xe0
 [  935.163579]  genl_family_rcv_msg_doit.isra.0+0xc2/0x110
 [  935.164086]  genl_rcv_msg+0x17d/0x2b0
 [  935.164460]  ? devlink_get_from_attrs_lock+0x110/0x110
 [  935.164936]  ? devlink_nl_cmd_eswitch_get_doit+0x290/0x290
 [  935.165436]  ? devlink_pernet_pre_exit+0xf0/0xf0
 [  935.165880]  ? genl_family_rcv_msg_doit.isra.0+0x110/0x110
 [  935.166381]  netlink_rcv_skb+0x54/0x100
 [  935.166769]  genl_rcv+0x24/0x40
 [  935.167109]  netlink_unicast+0x1f6/0x2c0
 [  935.167496]  netlink_sendmsg+0x239/0x4b0
 [  935.167901]  sock_sendmsg+0x38/0x60
 [  935.168265]  ? _copy_from_user+0x2a/0x60
 [  935.168654]  __sys_sendto+0x110/0x160
 [  935.169023]  ? handle_mm_fault+0xe4/0x270
 [  935.169422]  ? do_user_addr_fault+0x270/0x620
 [  935.169849]  __x64_sys_sendto+0x20/0x30
 [  935.170232]  do_syscall_64+0x3d/0x90
 [  935.170600]  entry_SYSCALL_64_after_hwframe+0x46/0xb0
 [  935.171075] RIP: 0033:0x7fa48dd1340a
 [  935.171438] Code: d8 64 89 02 48 c7 c0 ff ff ff ff eb b8 0f 1f 00 f3 0f 1e fa 41 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 15 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 7e c3 0f 1f 44 00 00 41 54 48 83 ec 30 44 89
 [  935.172982] RSP: 002b:00007ffcd3a8a498 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
 [  935.173683] RAX: ffffffffffffffda RBX: 00000000008eeb00 RCX: 00007fa48dd1340a
 [  935.174296] RDX: 0000000000000038 RSI: 00000000008eeb00 RDI: 0000000000000003
 [  935.174913] RBP: 00000000008ee910 R08: 00007fa48df37200 R09: 000000000000000c
 [  935.175532] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
 [  935.176166] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
 [  935.176790]  </TASK>
 [  935.177060] ---[ end trace 0000000000000000 ]---

Thanks

Powered by blists - more mailing lists