Date:   Fri, 1 Dec 2023 16:29:51 +0800
From:   Pengfei Xu <pengfei.xu@...el.com>
To:     syzbot <syzbot+ed812ed461471ab17a0c@...kaller.appspotmail.com>
CC:     "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
        "axboe@...nel.dk" <axboe@...nel.dk>,
        "dvyukov@...gle.com" <dvyukov@...gle.com>,
        "linux-block@...r.kernel.org" <linux-block@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-trace-kernel@...r.kernel.org" 
        <linux-trace-kernel@...r.kernel.org>,
        "mhiramat@...nel.org" <mhiramat@...nel.org>,
        "rostedt@...dmis.org" <rostedt@...dmis.org>,
        "syzkaller-bugs@...glegroups.com" <syzkaller-bugs@...glegroups.com>,
        <heng.su@...el.com>
Subject: Re: [syzbot] [block?] [trace?] INFO: task hung in blk_trace_ioctl (4)

Hi,

On 2023-12-01 at 05:17:19 +0800, syzbot wrote:
> Hello,
> 
> syzbot found the following issue on:
> 
> HEAD commit:    8c9660f65153 Add linux-next specific files for 20231124
> git tree:       linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=1006f178e80000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=ca1e8655505e280
> dashboard link: https://syzkaller.appspot.com/bug?extid=ed812ed461471ab17a0c
> compiler:       gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=14ec6e62e80000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=11964f7ce80000
> 
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/345ed4af3a0d/disk-8c9660f6.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/191053c69d57/vmlinux-8c9660f6.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/aac7ee5e55e0/bzImage-8c9660f6.xz
> 

  Thanks for your report!
  I reverted the commit 0d345996e4cb573f8cc81d49b3ee9a7fd2035bef on top of
  v6.3-rc7 mainline kernel, this issue was still reproduced.
  Anyway, it's an interesting problem, I will update the email if I find
  more clue of the problem.

  Best Regards,
  Thanks!

> The issue was bisected to:
> 
> commit 0d345996e4cb573f8cc81d49b3ee9a7fd2035bef
> Author: Pengfei Xu <pengfei.xu@...el.com>
> Date:   Mon Jul 31 03:04:18 2023 +0000
> 
>     x86/kernel: increase kcov coverage under arch/x86/kernel folder
> 
> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=14782eaae80000
> final oops:     https://syzkaller.appspot.com/x/report.txt?x=16782eaae80000
> console output: https://syzkaller.appspot.com/x/log.txt?x=12782eaae80000
> 
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+ed812ed461471ab17a0c@...kaller.appspotmail.com
> Fixes: 0d345996e4cb ("x86/kernel: increase kcov coverage under arch/x86/kernel folder")
> 
> INFO: task syz-executor216:5234 blocked for more than 143 seconds.
>       Not tainted 6.7.0-rc2-next-20231124-syzkaller #0
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> task:syz-executor216 state:D stack:29104 pid:5234  tgid:5232  ppid:5106   flags:0x00004006
> Call Trace:
>  <TASK>
>  context_switch kernel/sched/core.c:5399 [inline]
>  __schedule+0xf15/0x5c00 kernel/sched/core.c:6726
>  __schedule_loop kernel/sched/core.c:6801 [inline]
>  schedule+0xe7/0x270 kernel/sched/core.c:6816
>  schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6873
>  __mutex_lock_common kernel/locking/mutex.c:679 [inline]
>  __mutex_lock+0x5b4/0x9c0 kernel/locking/mutex.c:747
>  blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
>  blkdev_common_ioctl+0x1575/0x1ce0 block/ioctl.c:562
>  blkdev_ioctl+0x249/0x770 block/ioctl.c:627
>  vfs_ioctl fs/ioctl.c:51 [inline]
>  __do_sys_ioctl fs/ioctl.c:871 [inline]
>  __se_sys_ioctl fs/ioctl.c:857 [inline]
>  __x64_sys_ioctl+0x18f/0x210 fs/ioctl.c:857
>  do_syscall_x64 arch/x86/entry/common.c:51 [inline]
>  do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82
>  entry_SYSCALL_64_after_hwframe+0x62/0x6a
> RIP: 0033:0x7f914d80ca89
> RSP: 002b:00007f914cfa3168 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> RAX: ffffffffffffffda RBX: 00007f914d89c1d8 RCX: 00007f914d80ca89
> RDX: 0000000000000000 RSI: 0000000000001276 RDI: 0000000000000004
> RBP: 00007f914d89c1d0 R08: 00007ffdb3560fd7 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000000000246 R12: 00007f914d89c1dc
> R13: 000000000000006e R14: 00007ffdb3560ef0 R15: 00007ffdb3560fd8
>  </TASK>
> 
> Showing all locks held in the system:
> 1 lock held by khungtaskd/29:
>  #0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:301 [inline]
>  #0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
>  #0: ffffffff8cfacf60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6613
> 1 lock held by klogd/4501:
>  #0: ffff8880b993c718 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x29/0x130 kernel/sched/core.c:558
> 2 locks held by getty/4815:
>  #0: ffff8880273ee0a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
>  #1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc4/0x1490 drivers/tty/n_tty.c:2201
> 2 locks held by kworker/0:5/5180:
> 2 locks held by syz-executor216/5233:
> 1 lock held by syz-executor216/5234:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5240:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5241:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5245:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5246:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5248:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5249:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5251:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5252:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5254:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 1 lock held by syz-executor216/5255:
>  #0: ffff888142f94490 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_ioctl+0xc9/0x290 kernel/trace/blktrace.c:736
> 
> =============================================
> 
> NMI backtrace for cpu 0
> CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-rc2-next-20231124-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023
> Call Trace:
>  <TASK>
>  __dump_stack lib/dump_stack.c:88 [inline]
>  dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
>  nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
>  nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
>  trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
>  check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
>  watchdog+0xf86/0x1210 kernel/hung_task.c:379
>  kthread+0x2c1/0x3a0 kernel/kthread.c:389
>  ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
>  ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
>  </TASK>
> Sending NMI from CPU 0 to CPUs 1:
> NMI backtrace for cpu 1
> CPU: 1 PID: 5233 Comm: syz-executor216 Not tainted 6.7.0-rc2-next-20231124-syzkaller #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023
> RIP: 0010:__sanitizer_cov_trace_switch+0x4f/0x90 kernel/kcov.c:341
> Code: 83 f8 10 75 2f 41 bd 03 00 00 00 4c 8b 75 00 31 db 4d 85 f6 74 1e 48 8b 74 dd 10 4c 89 e2 4c 89 ef 48 83 c3 01 48 8b 4c 24 28 <e8> 2c fe ff ff 49 39 de 75 e2 5b 5d 41 5c 41 5d 41 5e c3 48 83 f8
> RSP: 0018:ffffc90003fdf3a0 EFLAGS: 00000202
> RAX: 0000000000000008 RBX: 0000000000000001 RCX: ffffffff813a44c8
> RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000001
> RBP: ffffffff8ac9d800 R08: 0000000000000001 R09: 0000000000000000
> R10: 0000000000000001 R11: dffffc0000000000 R12: 0000000000000001
> R13: 0000000000000001 R14: 0000000000000003 R15: 0000000000000001
> FS:  00007f914cfc46c0(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f914d899f18 CR3: 00000000a0ad4000 CR4: 00000000003506f0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  <NMI>
>  </NMI>
>  <TASK>
>  unwind_next_frame+0xea8/0x2390 arch/x86/kernel/unwind_orc.c:641
>  arch_stack_walk+0xfa/0x170 arch/x86/kernel/stacktrace.c:25
>  stack_trace_save+0x95/0xd0 kernel/stacktrace.c:122
>  save_stack+0x15e/0x1f0 mm/page_owner.c:130
>  __set_page_owner+0x1f/0x60 mm/page_owner.c:196
>  set_page_owner include/linux/page_owner.h:31 [inline]
>  post_alloc_hook+0x2d0/0x350 mm/page_alloc.c:1533
>  prep_new_page mm/page_alloc.c:1540 [inline]
>  get_page_from_freelist+0xa16/0x3690 mm/page_alloc.c:3344
>  __alloc_pages+0x22d/0x2400 mm/page_alloc.c:4600
>  alloc_pages_mpol+0x258/0x5f0 mm/mempolicy.c:2133
>  relay_alloc_buf kernel/relay.c:121 [inline]
>  relay_create_buf kernel/relay.c:162 [inline]
>  relay_open_buf.part.0+0x27d/0xba0 kernel/relay.c:384
>  relay_open_buf kernel/relay.c:536 [inline]
>  relay_open+0x641/0xab0 kernel/relay.c:517
>  do_blk_trace_setup+0x4a9/0xaa0 kernel/trace/blktrace.c:591
>  __blk_trace_setup+0xd8/0x180 kernel/trace/blktrace.c:631
>  blk_trace_ioctl+0x163/0x290 kernel/trace/blktrace.c:741
>  blkdev_ioctl+0x111/0x770 block/ioctl.c:622
>  vfs_ioctl fs/ioctl.c:51 [inline]
>  __do_sys_ioctl fs/ioctl.c:871 [inline]
>  __se_sys_ioctl fs/ioctl.c:857 [inline]
>  __x64_sys_ioctl+0x18f/0x210 fs/ioctl.c:857
>  do_syscall_x64 arch/x86/entry/common.c:51 [inline]
>  do_syscall_64+0x40/0x110 arch/x86/entry/common.c:82
>  entry_SYSCALL_64_after_hwframe+0x62/0x6a
> RIP: 0033:0x7f914d80ca89
> Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 1c 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007f914cfc4168 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> RAX: ffffffffffffffda RBX: 00007f914d89c1c8 RCX: 00007f914d80ca89
> RDX: 0000000020000380 RSI: 00000000c0481273 RDI: 0000000000000004
> RBP: 00007f914d89c1c0 R08: 00007f914cfc46c0 R09: 0000000000000000
> R10: 00007f914cfc46c0 R11: 0000000000000246 R12: 00007f914d89c1cc
> R13: 000000000000000b R14: 00007ffdb3560ef0 R15: 00007ffdb3560fd8
>  </TASK>
> INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 2.100 msecs
> 
> 
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@...glegroups.com.
> 
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> For information about bisection process see: https://goo.gl/tpsmEJ#bisection
> 
> If the report is already addressed, let syzbot know by replying with:
> #syz fix: exact-commit-title
> 
> If you want syzbot to run the reproducer, reply with:
> #syz test: git://repo/address.git branch-or-commit-hash
> If you attach or paste a git patch, syzbot will apply it before testing.
> 
> If you want to overwrite report's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
> 
> If the report is a duplicate of another one, reply with:
> #syz dup: exact-subject-of-another-report
> 
> If you want to undo deduplication, reply with:
> #syz undup

Powered by blists - more mailing lists