netdev - Re: [syzbot] [net?] WARNING: suspicious RCU usage in dev

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CANn89iJFATii0PdPxfhjurHXiBo6j5Dgqunok1dLJw_NNYtb7g@mail.gmail.com>
Date: Wed, 3 Jul 2024 12:10:52 +0200
From: Eric Dumazet <edumazet@...gle.com>
To: syzbot <syzbot+2120b9a8f96b3fa90bad@...kaller.appspotmail.com>, 
	Johannes Berg <johannes.berg@...el.com>
Cc: davem@...emloft.net, jhs@...atatu.com, jiri@...nulli.us, kuba@...nel.org, 
	linux-kernel@...r.kernel.org, netdev@...r.kernel.org, pabeni@...hat.com, 
	syzkaller-bugs@...glegroups.com, xiyou.wangcong@...il.com
Subject: Re: [syzbot] [net?] WARNING: suspicious RCU usage in dev_activate

On Wed, Jul 3, 2024 at 12:07 PM syzbot
<syzbot+2120b9a8f96b3fa90bad@...kaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit:    1dfe225e9af5 Merge tag 'scsi-fixes' of git://git.kernel.or..
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=14a4f2d1980000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=1ace69f521989b1f
> dashboard link: https://syzkaller.appspot.com/bug?extid=2120b9a8f96b3fa90bad
> compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
>
> Unfortunately, I don't have any reproducer for this issue yet.
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/3894cf8b5271/disk-1dfe225e.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/573c202ade8f/vmlinux-1dfe225e.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/a356d869b8f3/bzImage-1dfe225e.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+2120b9a8f96b3fa90bad@...kaller.appspotmail.com
>
> =============================
> WARNING: suspicious RCU usage
> 6.10.0-rc6-syzkaller-00051-g1dfe225e9af5 #0 Not tainted
> -----------------------------
> net/sched/sch_generic.c:1249 suspicious rcu_dereference_protected() usage!
>
> other info that might help us debug this:
>
>
> rcu_scheduler_active = 2, debug_locks = 1
> 3 locks held by kworker/u8:0/11:
>  #0: ffff88801efaa148 ((wq_completion)bond0#9){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3223 [inline]
>  #0: ffff88801efaa148 ((wq_completion)bond0#9){+.+.}-{0:0}, at: process_scheduled_works+0x90a/0x1830 kernel/workqueue.c:3329
>  #1: ffffc90000107d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3224 [inline]
>  #1: ffffc90000107d00 ((work_completion)(&(&bond->mii_work)->work)){+.+.}-{0:0}, at: process_scheduled_works+0x945/0x1830 kernel/workqueue.c:3329
>  #2: ffffffff8e333f20 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:329 [inline]
>  #2: ffffffff8e333f20 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:781 [inline]
>  #2: ffffffff8e333f20 (rcu_read_lock){....}-{1:2}, at: bond_mii_monitor+0x174/0x3170 drivers/net/bonding/bond_main.c:2824
>
> stack backtrace:
> CPU: 0 PID: 11 Comm: kworker/u8:0 Not tainted 6.10.0-rc6-syzkaller-00051-g1dfe225e9af5 #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
> Workqueue: bond0 bond_mii_monitor
> Call Trace:
>  <TASK>
>  __dump_stack lib/dump_stack.c:88 [inline]
>  dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
>  lockdep_rcu_suspicious+0x221/0x340 kernel/locking/lockdep.c:6712
>  dev_activate+0xf8/0x1240 net/sched/sch_generic.c:1249
>  linkwatch_do_dev+0xfb/0x170 net/core/link_watch.c:173
>  ethtool_op_get_link+0x15/0x60 net/ethtool/ioctl.c:62
>  bond_check_dev_link+0x1f1/0x3f0 drivers/net/bonding/bond_main.c:757
>  bond_miimon_inspect drivers/net/bonding/bond_main.c:2604 [inline]
>  bond_mii_monitor+0x49a/0x3170 drivers/net/bonding/bond_main.c:2826
>  process_one_work kernel/workqueue.c:3248 [inline]
>  process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3329
>  worker_thread+0x86d/0xd50 kernel/workqueue.c:3409
>  kthread+0x
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@...glegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the report is already addressed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want to overwrite report's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the report is a duplicate of another one, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup

I think this came with this patch :

commit facd15dfd69122042502d99ab8c9f888b48ee994
Author: Johannes Berg <johannes.berg@...el.com>
Date:   Mon Dec 4 21:47:07 2023 +0100

    net: core: synchronize link-watch when carrier is queried


Issue here is that ethtool_op_get_link() could be called from RCU contexts.

Adding linkwatch_sync_dev() in it broke this case.

BTW, this commit also made it difficult to convert "ip link" dumps to
not use RTNL, but rely on RCU instead.