lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <202305161035.aee940eb-oliver.sang@intel.com>
Date:   Tue, 16 May 2023 11:04:20 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Dominique Martinet <asmadeus@...ewreck.org>
CC:     <oe-lkp@...ts.linux.dev>, <lkp@...el.com>,
        <linux-fsdevel@...r.kernel.org>,
        Alexander Viro <viro@...iv.linux.org.uk>,
        Christian Brauner <brauner@...nel.org>,
        Jens Axboe <axboe@...nel.dk>,
        "Pavel Begunkov" <asml.silence@...il.com>,
        Stefan Roesch <shr@...com>, Clay Harris <bugs@...ycon.org>,
        Dave Chinner <david@...morbit.com>,
        <linux-kernel@...r.kernel.org>, <io-uring@...r.kernel.org>,
        "Dominique Martinet" <asmadeus@...ewreck.org>,
        <oliver.sang@...el.com>
Subject: Re: [PATCH v2 4/6] kernfs: implement readdir FMODE_NOWAIT


Hello,

kernel test robot noticed "INFO:task_blocked_for_more_than#seconds" on:

commit: a551138c4b3b9fd7e74a55d6074a013bab6246db ("[PATCH v2 4/6] kernfs: implement readdir FMODE_NOWAIT")
url: https://github.com/intel-lab-lkp/linux/commits/Dominique-Martinet/fs-split-off-vfs_getdents-function-of-getdents64-syscall/20230510-185542
patch link: https://lore.kernel.org/all/20230422-uring-getdents-v2-4-2db1e37dc55e@codewreck.org/
patch subject: [PATCH v2 4/6] kernfs: implement readdir FMODE_NOWAIT

in testcase: boot

compiler: gcc-11
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

(please refer to attached dmesg/kmsg for entire log/backtrace)



If you fix the issue, kindly add following tag
| Reported-by: kernel test robot <oliver.sang@...el.com>
| Link: https://lore.kernel.org/oe-lkp/202305161035.aee940eb-oliver.sang@intel.com


[  989.795384][   T32] INFO: task systemd:1 blocked for more than 491 seconds.
[  989.797105][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  989.798652][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  989.805157][   T32] task:systemd         state:D stack:0     pid:1     ppid:0      flags:0x00000000
[  989.807376][   T32] Call Trace:
[  989.807873][   T32]  <TASK>
[ 989.808312][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 989.809423][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 989.810300][ T32] ? is_bpf_text_address (arch/x86/include/asm/preempt.h:85 include/linux/rcupdate.h:99 include/linux/rcupdate.h:805 kernel/bpf/core.c:721) 
[ 989.812634][ T32] ? __kernel_text_address (kernel/extable.c:79) 
[ 989.813489][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:26) 
[ 989.814106][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 989.814806][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 989.815384][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 989.816060][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073) 
[ 989.816862][ T32] ? filter_irq_stacks (kernel/stacktrace.c:114) 
[ 989.817552][ T32] ? down_write_killable (kernel/locking/rwsem.c:997) 
[ 989.818224][ T32] ? kasan_save_stack (mm/kasan/common.c:46) 
[ 989.818982][ T32] ? kasan_set_track (mm/kasan/common.c:52) 
[ 989.820158][ T32] ? do_filp_open (fs/namei.c:601 fs/namei.c:612 fs/namei.c:3817) 
[ 989.821366][ T32] ? do_sys_openat2 (fs/open.c:1356) 
[ 989.822619][ T32] ? __x64_sys_openat (fs/open.c:1383) 
[ 989.823954][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 989.825170][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 989.826651][ T32] down_read (kernel/locking/rwsem.c:1518) 
[ 989.827317][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518) 
[ 989.828033][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86) 
[ 989.828720][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265) 
[ 989.829326][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135) 
[ 989.830424][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651) 
[ 989.831513][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294) 
[ 989.832733][ T32] walk_component (fs/namei.c:1994) 
[ 989.833602][ T32] link_path_walk+0x533/0xa00 
[ 989.834386][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243) 
[ 989.835107][ T32] ? __mutex_init (arch/x86/include/asm/atomic.h:41 include/linux/atomic/atomic-instrumented.h:42 include/linux/osq_lock.h:30 kernel/locking/mutex.c:52) 
[ 989.835787][ T32] ? __alloc_file (fs/file_table.c:154) 
[ 989.836454][ T32] path_openat (fs/namei.c:2250 (discriminator 2) fs/namei.c:3787 (discriminator 2)) 
[ 989.837123][ T32] ? vfs_tmpfile_open (fs/namei.c:3773) 
[ 989.837759][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 989.838514][ T32] do_filp_open (fs/namei.c:3818) 
[ 989.839286][ T32] ? __update_load_avg_se (kernel/sched/pelt.c:118 kernel/sched/pelt.c:226 kernel/sched/pelt.c:308) 
[ 989.840569][ T32] ? may_open_dev (fs/namei.c:3812) 
[ 989.841576][ T32] ? update_load_avg (kernel/sched/fair.c:3920 kernel/sched/fair.c:4255) 
[ 989.842373][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.843492][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153) 
[ 989.844732][ T32] ? alloc_fd (arch/x86/include/asm/bitops.h:68 include/asm-generic/bitops/instrumented-non-atomic.h:29 fs/file.c:251 fs/file.c:540) 
[ 989.845759][ T32] do_sys_openat2 (fs/open.c:1356) 
[ 989.846384][ T32] ? build_open_flags (fs/open.c:1342) 
[ 989.847049][ T32] __x64_sys_openat (fs/open.c:1383) 
[ 989.847689][ T32] ? __ia32_compat_sys_open (fs/open.c:1383) 
[ 989.848385][ T32] ? schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 989.848957][ T32] ? switch_fpu_return (arch/x86/include/asm/bitops.h:75 include/asm-generic/bitops/instrumented-atomic.h:42 include/linux/thread_info.h:94 arch/x86/kernel/fpu/context.h:80 arch/x86/kernel/fpu/core.c:752) 
[ 989.849627][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 989.850219][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  989.850964][   T32] RIP: 0033:0x7f5c316f8be7
[  989.851562][   T32] RSP: 002b:00007ffc354a1f10 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
[  989.852586][   T32] RAX: ffffffffffffffda RBX: 0000562309e04d60 RCX: 00007f5c316f8be7
[  989.853582][   T32] RDX: 0000000000080000 RSI: 0000562309eb1320 RDI: 00000000ffffff9c
[  989.854556][   T32] RBP: 0000562309eb1320 R08: 0000000000000008 R09: 0000000000000001
[  989.855573][   T32] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000080000
[  989.856562][   T32] R13: 0000562309e04d60 R14: 0000000000000001 R15: 00007ffc354a2320
[  989.857561][   T32]  </TASK>
[  989.858016][   T32] INFO: task modprobe:90 blocked for more than 491 seconds.
[  989.859063][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  989.859862][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  989.861044][   T32] task:modprobe        state:D stack:0     pid:90    ppid:1      flags:0x00004002
[  989.862166][   T32] Call Trace:
[  989.862649][   T32]  <TASK>
[ 989.863098][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 989.863704][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 989.864384][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 989.865125][ T32] ? idr_get_free (arch/x86/include/asm/bitops.h:228 arch/x86/include/asm/bitops.h:240 include/asm-generic/bitops/instrumented-non-atomic.h:142 lib/radix-tree.c:113 lib/radix-tree.c:1518) 
[ 989.865743][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 989.866406][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 989.866974][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 989.867669][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180) 
[ 989.868395][ T32] ? down_timeout (kernel/locking/rwsem.c:1108) 
[ 989.869015][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.869640][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651) 
[ 989.870566][ T32] ? stack_trace_save (kernel/stacktrace.c:123) 
[ 989.871721][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574) 
[ 989.872459][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571) 
[ 989.873626][ T32] ? kasan_save_stack (mm/kasan/common.c:47) 
[ 989.874858][ T32] ? kasan_save_stack (mm/kasan/common.c:46) 
[ 989.876064][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754) 
[ 989.876931][ T32] ? pcpu_chunk_refresh_hint (mm/percpu-internal.h:114 (discriminator 3) mm/percpu.c:762 (discriminator 3)) 
[ 989.877791][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044) 
[ 989.878954][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61) 
[ 989.879719][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41) 
[ 989.880421][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.881343][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986) 
[ 989.882220][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233) 
[ 989.882902][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451) 
[ 989.883569][ T32] ? kobject_create_and_add (lib/kobject.c:444) 
[ 989.884262][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.885122][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153) 
[ 989.885929][ T32] ? srcu_module_notify (kernel/rcu/srcutree.c:1921 kernel/rcu/srcutree.c:1954) 
[ 989.889953][ T32] ? tracepoint_module_notify (kernel/tracepoint.c:664 kernel/tracepoint.c:709 kernel/tracepoint.c:701) 
[ 989.890685][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377) 
[ 989.891350][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374) 
[ 989.892053][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343) 
[ 989.892766][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296) 
[ 989.893417][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945) 
[ 989.894035][ T32] load_module (kernel/module/main.c:2965) 
[ 989.894644][ T32] ? post_relocation (kernel/module/main.c:2829) 
[ 989.895343][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38) 
[ 989.896301][ T32] ? __cond_resched (kernel/sched/core.c:8533) 
[ 989.897515][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099) 
[ 989.903046][ T32] __do_sys_finit_module (kernel/module/main.c:3099) 
[ 989.904404][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061) 
[ 989.905751][ T32] ? randomize_page (mm/util.c:533) 
[ 989.906912][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445) 
[ 989.907570][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 989.908151][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  989.908891][   T32] RIP: 0033:0x7feb8112a9b9
[  989.909479][   T32] RSP: 002b:00007ffd74170a98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  989.910515][   T32] RAX: ffffffffffffffda RBX: 000055ac05343d30 RCX: 00007feb8112a9b9
[  989.911580][   T32] RDX: 0000000000000000 RSI: 000055ac05277260 RDI: 0000000000000003
[  989.912567][   T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 000055ac053458b0
[  989.913550][   T32] R10: 0000000000000003 R11: 0000000000000246 R12: 000055ac05277260
[  989.914742][   T32] R13: 0000000000000000 R14: 000055ac05343cb0 R15: 000055ac05343d30
[  989.915789][   T32]  </TASK>
[  989.916463][   T32] INFO: task modprobe:91 blocked for more than 491 seconds.
[  989.917554][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  989.918474][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  989.920351][   T32] task:modprobe        state:D stack:0     pid:91    ppid:1      flags:0x00004002
[  989.922592][   T32] Call Trace:
[  989.923606][   T32]  <TASK>
[ 989.924503][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 989.925717][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 989.926964][ T32] ? osq_unlock (kernel/locking/osq_lock.c:22 kernel/locking/osq_lock.c:210) 
[ 989.927574][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 989.928128][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 989.928826][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180) 
[ 989.929533][ T32] ? down_timeout (kernel/locking/rwsem.c:1108) 
[ 989.930115][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.930743][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651) 
[ 989.931621][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574) 
[ 989.932705][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571) 
[ 989.933976][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 989.935249][ T32] ? notifier_call_chain (kernel/notifier.c:95) 
[ 989.936333][ T32] ? blocking_notifier_call_chain_robust (kernel/notifier.c:129 kernel/notifier.c:353 kernel/notifier.c:341) 
[ 989.937360][ T32] ? load_module (include/linux/notifier.h:209 kernel/module/main.c:2764 kernel/module/main.c:2945) 
[ 989.938087][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099) 
[ 989.938892][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 989.939627][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 989.940609][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754) 
[ 989.941241][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044) 
[ 989.942082][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61) 
[ 989.942885][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41) 
[ 989.943803][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.944568][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986) 
[ 989.945448][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233) 
[ 989.946551][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451) 
[ 989.947309][ T32] ? kobject_create_and_add (lib/kobject.c:444) 
[ 989.948025][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 989.948665][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153) 
[ 989.949347][ T32] ? ddebug_module_notify (lib/dynamic_debug.c:1344 lib/dynamic_debug.c:1336) 
[ 989.950021][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377) 
[ 989.950943][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374) 
[ 989.952014][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343) 
[ 989.953224][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296) 
[ 989.954395][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945) 
[ 989.955507][ T32] load_module (kernel/module/main.c:2965) 
[ 989.956552][ T32] ? post_relocation (kernel/module/main.c:2829) 
[ 989.957756][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38) 
[ 989.958858][ T32] ? __cond_resched (kernel/sched/core.c:8533) 
[ 989.959756][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099) 
[ 989.961072][ T32] __do_sys_finit_module (kernel/module/main.c:3099) 
[ 989.962177][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061) 
[ 989.963144][ T32] ? randomize_page (mm/util.c:533) 
[ 989.963862][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445) 
[ 989.964664][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 989.965454][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  989.966270][   T32] RIP: 0033:0x7f1d905f79b9
[  989.967030][   T32] RSP: 002b:00007ffc9133bbd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  989.968331][   T32] RAX: ffffffffffffffda RBX: 00005651b1b1ae20 RCX: 00007f1d905f79b9
[  989.969548][   T32] RDX: 0000000000000000 RSI: 00005651b1622260 RDI: 0000000000000003
[  989.970830][   T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 00005651b1b1c8b0
[  989.972058][   T32] R10: 0000000000000003 R11: 0000000000000246 R12: 00005651b1622260
[  989.973341][   T32] R13: 0000000000000000 R14: 00005651b1b1af50 R15: 00005651b1b1ae20
[  989.974561][   T32]  </TASK>
[  989.975037][   T32] INFO: task systemd-journal:92 blocked for more than 491 seconds.
[  989.976285][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  989.977338][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  989.979000][   T32] task:systemd-journal state:D stack:0     pid:92    ppid:1      flags:0x00000000
[  989.981119][   T32] Call Trace:
[  989.982020][   T32]  <TASK>
[ 989.982888][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 989.983793][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 989.984786][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620) 
[ 989.985459][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24) 
[ 989.986085][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 989.986828][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 989.987663][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 989.988361][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073) 
[ 989.989399][ T32] ? down_write_killable (kernel/locking/rwsem.c:997) 
[ 989.990086][ T32] ? file_fdatawait_range (mm/filemap.c:3485) 
[ 989.990921][ T32] down_read (kernel/locking/rwsem.c:1518) 
[ 989.991817][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518) 
[ 989.993060][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86) 
[ 989.994335][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265) 
[ 989.995528][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135) 
[ 989.996846][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651) 
[ 989.997996][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294) 
[ 990.000155][ T32] walk_component (fs/namei.c:1994) 
[ 990.001367][ T32] link_path_walk+0x533/0xa00 
[ 990.002933][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243) 
[ 990.004345][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2)) 
[ 990.005493][ T32] filename_lookup (fs/namei.c:2509) 
[ 990.006694][ T32] ? may_linkat (fs/namei.c:2502) 
[ 990.007891][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138) 
[ 990.009149][ T32] user_path_at_empty (fs/namei.c:2909) 
[ 990.010387][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103) 
[ 990.011522][ T32] ? __ia32_sys_ustat (fs/statfs.c:98) 
[ 990.012771][ T32] __do_sys_statfs (fs/statfs.c:196) 
[ 990.013989][ T32] ? user_statfs (fs/statfs.c:193) 
[ 990.015146][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616) 
[ 990.015775][ T32] ? syscall_trace_enter+0x96/0x190 
[ 990.016557][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 990.017147][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  990.017918][   T32] RIP: 0033:0x7fd5039e98c7
[  990.018530][   T32] RSP: 002b:00007ffd5f26e0b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[  990.019615][   T32] RAX: ffffffffffffffda RBX: 00007ffd5f26e130 RCX: 00007fd5039e98c7
[  990.020612][   T32] RDX: 00007fd503aea1a4 RSI: 00007ffd5f26e130 RDI: 00007fd503ae0843
[  990.021627][   T32] RBP: 00007fd503ae0843 R08: 0000000000000000 R09: 0000000000000000
[  990.022622][   T32] R10: 00000000000002b0 R11: 0000000000000246 R12: 00007ffd5f26e2b8
[  990.023667][   T32] R13: 00007ffd5f26e2c8 R14: 00007fd503ae8610 R15: 0000000000000000
[  990.024655][   T32]  </TASK>
[  990.025154][   T32] INFO: task systemd-modules:93 blocked for more than 491 seconds.
[  990.027363][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  990.028884][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  990.030964][   T32] task:systemd-modules state:D stack:0     pid:93    ppid:1      flags:0x00000000
[  990.032126][   T32] Call Trace:
[  990.032627][   T32]  <TASK>
[ 990.033062][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 990.033683][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 990.034550][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620) 
[ 990.035252][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24) 
[ 990.035871][ T32] ? is_bpf_text_address (arch/x86/include/asm/preempt.h:85 include/linux/rcupdate.h:99 include/linux/rcupdate.h:805 kernel/bpf/core.c:721) 
[ 990.036528][ T32] ? kernel_text_address (kernel/extable.c:97 kernel/extable.c:94) 
[ 990.037170][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 990.038056][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 990.038750][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 990.039811][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073) 
[ 990.040514][ T32] ? down_write_killable (kernel/locking/rwsem.c:997) 
[ 990.041320][ T32] ? file_fdatawait_range (mm/filemap.c:3485) 
[ 990.042565][ T32] down_read (kernel/locking/rwsem.c:1518) 
[ 990.043578][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518) 
[ 990.044897][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86) 
[ 990.046130][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265) 
[ 990.053684][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135) 
[ 990.055053][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651) 
[ 990.056152][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294) 
[ 990.057463][ T32] walk_component (fs/namei.c:1994) 
[ 990.058592][ T32] link_path_walk+0x533/0xa00 
[ 990.060119][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243) 
[ 990.061518][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2)) 
[ 990.062658][ T32] filename_lookup (fs/namei.c:2509) 
[ 990.063775][ T32] ? may_linkat (fs/namei.c:2502) 
[ 990.064820][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138) 
[ 990.066006][ T32] user_path_at_empty (fs/namei.c:2909) 
[ 990.067157][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103) 
[ 990.068186][ T32] ? __ia32_sys_ustat (fs/statfs.c:98) 
[ 990.069355][ T32] __do_sys_statfs (fs/statfs.c:196) 
[ 990.070342][ T32] ? user_statfs (fs/statfs.c:193) 
[ 990.071166][ T32] ? mm_account_fault (arch/x86/include/asm/irqflags.h:134 include/linux/memcontrol.h:1079 include/linux/memcontrol.h:1111 include/linux/memcontrol.h:1100 mm/memory.c:5123) 
[ 990.071937][ T32] ? handle_mm_fault (mm/memory.c:5262) 
[ 990.072580][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616) 
[ 990.073128][ T32] ? do_user_addr_fault (include/linux/mmap_lock.h:170 arch/x86/mm/fault.c:1468) 
[ 990.073798][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 990.074387][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  990.075287][   T32] RIP: 0033:0x7f1b978518c7
[  990.075911][   T32] RSP: 002b:00007ffdb557fc48 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[  990.077454][   T32] RAX: ffffffffffffffda RBX: 00007ffdb557fcc0 RCX: 00007f1b978518c7
[  990.078536][   T32] RDX: 00007f1b974d11a4 RSI: 00007ffdb557fcc0 RDI: 00007f1b974c7843
[  990.079636][   T32] RBP: 00007f1b974c7843 R08: 0000000000000000 R09: 9fb8220300000000
[  990.080743][   T32] R10: 00000000478bfbff R11: 0000000000000246 R12: 00007ffdb557fe48
[  990.081743][   T32] R13: 00007ffdb557fe58 R14: 00007f1b974cf610 R15: 0000000000000000
[  990.082740][   T32]  </TASK>
[  990.083412][   T32] INFO: task systemd-remount:94 blocked for more than 491 seconds.
[  990.085248][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  990.086674][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  990.089254][   T32] task:systemd-remount state:D stack:0     pid:94    ppid:1      flags:0x00000000
[  990.091522][   T32] Call Trace:
[  990.092433][   T32]  <TASK>
[ 990.093316][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 990.094513][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 990.095872][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620) 
[ 990.097243][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24) 
[ 990.098489][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 990.099700][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 990.100802][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 990.102163][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073) 
[ 990.103566][ T32] ? down_write_killable (kernel/locking/rwsem.c:997) 
[ 990.104932][ T32] ? file_fdatawait_range (mm/filemap.c:3485) 
[ 990.106279][ T32] down_read (kernel/locking/rwsem.c:1518) 
[ 990.107433][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518) 
[ 990.108819][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86) 
[ 990.110093][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265) 
[ 990.111095][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135) 
[ 990.111862][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651) 
[ 990.112824][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294) 
[ 990.114078][ T32] walk_component (fs/namei.c:1994) 
[ 990.114725][ T32] link_path_walk+0x533/0xa00 
[ 990.116148][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243) 
[ 990.117495][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2)) 
[ 990.118168][ T32] filename_lookup (fs/namei.c:2509) 
[ 990.118831][ T32] ? may_linkat (fs/namei.c:2502) 
[ 990.119593][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138) 
[ 990.120820][ T32] user_path_at_empty (fs/namei.c:2909) 
[ 990.121746][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103) 
[ 990.122653][ T32] ? __ia32_sys_ustat (fs/statfs.c:98) 
[ 990.123499][ T32] __do_sys_statfs (fs/statfs.c:196) 
[ 990.124588][ T32] ? user_statfs (fs/statfs.c:193) 
[ 990.125742][ T32] ? mm_account_fault (arch/x86/include/asm/irqflags.h:134 include/linux/memcontrol.h:1079 include/linux/memcontrol.h:1111 include/linux/memcontrol.h:1100 mm/memory.c:5123) 
[ 990.126897][ T32] ? handle_mm_fault (mm/memory.c:5262) 
[ 990.128044][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616) 
[ 990.129025][ T32] ? do_user_addr_fault (include/linux/mmap_lock.h:170 arch/x86/mm/fault.c:1468) 
[ 990.130186][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 990.131288][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  990.132597][   T32] RIP: 0033:0x7f2eda3488c7
[  990.133566][   T32] RSP: 002b:00007ffce4e63338 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[  990.135381][   T32] RAX: ffffffffffffffda RBX: 00007ffce4e633b0 RCX: 00007f2eda3488c7
[  990.137068][   T32] RDX: 00007f2ed9fac1a4 RSI: 00007ffce4e633b0 RDI: 00007f2ed9fa2843
[  990.138642][   T32] RBP: 00007f2ed9fa2843 R08: 0000000000000000 R09: 0000000000000000
[  990.140356][   T32] R10: 00000000000002b0 R11: 0000000000000246 R12: 00007ffce4e63538
[  990.142047][   T32] R13: 00007ffce4e63548 R14: 00007f2ed9faa610 R15: 0000000000000000
[  990.143852][   T32]  </TASK>
[  990.144662][   T32] INFO: task udevadm:95 blocked for more than 491 seconds.
[  990.146198][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[  990.147520][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  990.149377][   T32] task:udevadm         state:D stack:0     pid:95    ppid:1      flags:0x00000000
[  990.151330][   T32] Call Trace:
[  990.152142][   T32]  <TASK>
[ 990.152910][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 990.153969][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 990.155177][ T32] ? kernel_text_address (kernel/extable.c:97 kernel/extable.c:94) 
[ 990.156388][ T32] ? unwind_next_frame (arch/x86/kernel/unwind_orc.c:378 arch/x86/kernel/unwind_orc.c:620) 
[ 990.157575][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:24) 
[ 990.158682][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 990.159875][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 990.160867][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 990.162097][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073) 
[ 990.163373][ T32] ? down_write_killable (kernel/locking/rwsem.c:997) 
[ 990.164566][ T32] ? file_fdatawait_range (mm/filemap.c:3485) 
[ 990.165743][ T32] down_read (kernel/locking/rwsem.c:1518) 
[ 990.166742][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518) 
[ 990.168036][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86) 
[ 990.169190][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265) 
[ 990.170276][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135) 
[ 990.171430][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651) 
[ 990.172459][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294) 
[ 990.173637][ T32] walk_component (fs/namei.c:1994) 
[ 990.174710][ T32] link_path_walk+0x533/0xa00 
[ 990.176103][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243) 
[ 990.177383][ T32] path_lookupat (fs/namei.c:2248 (discriminator 2) fs/namei.c:2478 (discriminator 2)) 
[ 990.178528][ T32] filename_lookup (fs/namei.c:2509) 
[ 990.179749][ T32] ? may_linkat (fs/namei.c:2502) 
[ 990.180834][ T32] ? strncpy_from_user (arch/x86/include/asm/uaccess.h:605 lib/strncpy_from_user.c:138) 
[ 990.181998][ T32] user_path_at_empty (fs/namei.c:2909) 
[ 990.183148][ T32] user_statfs (include/linux/namei.h:57 fs/statfs.c:103) 
[ 990.184197][ T32] ? __ia32_sys_ustat (fs/statfs.c:98) 
[ 990.185340][ T32] __do_sys_statfs (fs/statfs.c:196) 
[ 990.186430][ T32] ? user_statfs (fs/statfs.c:193) 
[ 990.187538][ T32] ? mm_account_fault (arch/x86/include/asm/irqflags.h:134 include/linux/memcontrol.h:1079 include/linux/memcontrol.h:1111 include/linux/memcontrol.h:1100 mm/memory.c:5123) 
[ 990.188691][ T32] ? handle_mm_fault (mm/memory.c:5262) 
[ 990.189867][ T32] ? up_read (arch/x86/include/asm/atomic64_64.h:160 include/linux/atomic/atomic-long.h:71 include/linux/atomic/atomic-instrumented.h:1362 kernel/locking/rwsem.c:1347 kernel/locking/rwsem.c:1616) 
[ 990.190867][ T32] ? do_user_addr_fault (include/linux/mmap_lock.h:170 arch/x86/mm/fault.c:1468) 
[ 990.192059][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 990.193125][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[  990.194455][   T32] RIP: 0033:0x7f7e23ebf8c7
[  990.195521][   T32] RSP: 002b:00007ffe9f551448 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
[  990.197388][   T32] RAX: ffffffffffffffda RBX: 00007ffe9f5514c0 RCX: 00007f7e23ebf8c7
[  990.199194][   T32] RDX: 00007f7e23fe21a4 RSI: 00007ffe9f5514c0 RDI: 00007f7e23fd8843
[  990.201122][   T32] RBP: 00007f7e23fd8843 R08: 0000000000000000 R09: 9fb8220300000000
[  990.207119][   T32] R10: 00000000478bfbff R11: 0000000000000246 R12: 00007ffe9f551648
[  990.208948][   T32] R13: 00007ffe9f551670 R14: 00007f7e23fe0610 R15: 0000000000000000
[  990.210742][   T32]  </TASK>
[ 1481.315358][   T32] INFO: task systemd:1 blocked for more than 983 seconds.
[ 1481.316989][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 1481.318380][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1481.320287][   T32] task:systemd         state:D stack:0     pid:1     ppid:0      flags:0x00000000
[ 1481.322300][   T32] Call Trace:
[ 1481.323089][   T32]  <TASK>
[ 1481.323722][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 1481.324790][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 1481.325948][ T32] ? is_bpf_text_address (arch/x86/include/asm/preempt.h:85 include/linux/rcupdate.h:99 include/linux/rcupdate.h:805 kernel/bpf/core.c:721) 
[ 1481.327112][ T32] ? __kernel_text_address (kernel/extable.c:79) 
[ 1481.332555][ T32] ? arch_stack_walk (arch/x86/kernel/stacktrace.c:26) 
[ 1481.333657][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 1481.334800][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 1481.335755][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 1481.337010][ T32] rwsem_down_read_slowpath (kernel/locking/rwsem.c:1073) 
[ 1481.338251][ T32] ? filter_irq_stacks (kernel/stacktrace.c:114) 
[ 1481.339367][ T32] ? down_write_killable (kernel/locking/rwsem.c:997) 
[ 1481.340483][ T32] ? kasan_save_stack (mm/kasan/common.c:46) 
[ 1481.341479][ T32] ? kasan_set_track (mm/kasan/common.c:52) 
[ 1481.342515][ T32] ? do_filp_open (fs/namei.c:601 fs/namei.c:612 fs/namei.c:3817) 
[ 1481.343547][ T32] ? do_sys_openat2 (fs/open.c:1356) 
[ 1481.344644][ T32] ? __x64_sys_openat (fs/open.c:1383) 
[ 1481.345757][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 1481.346788][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.348135][ T32] down_read (kernel/locking/rwsem.c:1518) 
[ 1481.349175][ T32] ? rwsem_down_read_slowpath (kernel/locking/rwsem.c:1518) 
[ 1481.350421][ T32] ? read_word_at_a_time (include/asm-generic/rwonce.h:86) 
[ 1481.357275][ T32] ? d_same_name (arch/x86/include/asm/word-at-a-time.h:84 fs/dcache.c:227 fs/dcache.c:278 fs/dcache.c:2265) 
[ 1481.358309][ T32] kernfs_dop_revalidate (include/linux/instrumented.h:68 include/linux/atomic/atomic-instrumented.h:27 fs/kernfs/dir.c:36 fs/kernfs/dir.c:42 fs/kernfs/dir.c:1135) 
[ 1481.359461][ T32] lookup_fast (fs/namei.c:859 fs/namei.c:856 fs/namei.c:1651) 
[ 1481.360441][ T32] ? kernfs_iop_permission (fs/kernfs/inode.c:294) 
[ 1481.361528][ T32] walk_component (fs/namei.c:1994) 
[ 1481.362501][ T32] link_path_walk+0x533/0xa00 
[ 1481.363800][ T32] ? lookup_one_len_unlocked (fs/namei.c:2243) 
[ 1481.365048][ T32] ? __mutex_init (arch/x86/include/asm/atomic.h:41 include/linux/atomic/atomic-instrumented.h:42 include/linux/osq_lock.h:30 kernel/locking/mutex.c:52) 
[ 1481.366100][ T32] ? __alloc_file (fs/file_table.c:154) 
[ 1481.367082][ T32] path_openat (fs/namei.c:2250 (discriminator 2) fs/namei.c:3787 (discriminator 2)) 
[ 1481.368064][ T32] ? vfs_tmpfile_open (fs/namei.c:3773) 
[ 1481.369074][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.370313][ T32] do_filp_open (fs/namei.c:3818) 
[ 1481.371266][ T32] ? __update_load_avg_se (kernel/sched/pelt.c:118 kernel/sched/pelt.c:226 kernel/sched/pelt.c:308) 
[ 1481.372368][ T32] ? may_open_dev (fs/namei.c:3812) 
[ 1481.373365][ T32] ? update_load_avg (kernel/sched/fair.c:3920 kernel/sched/fair.c:4255) 
[ 1481.374450][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.375378][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153) 
[ 1481.376371][ T32] ? alloc_fd (arch/x86/include/asm/bitops.h:68 include/asm-generic/bitops/instrumented-non-atomic.h:29 fs/file.c:251 fs/file.c:540) 
[ 1481.377250][ T32] do_sys_openat2 (fs/open.c:1356) 
[ 1481.378142][ T32] ? build_open_flags (fs/open.c:1342) 
[ 1481.379113][ T32] __x64_sys_openat (fs/open.c:1383) 
[ 1481.380196][ T32] ? __ia32_compat_sys_open (fs/open.c:1383) 
[ 1481.381372][ T32] ? schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 1481.382397][ T32] ? switch_fpu_return (arch/x86/include/asm/bitops.h:75 include/asm-generic/bitops/instrumented-atomic.h:42 include/linux/thread_info.h:94 arch/x86/kernel/fpu/context.h:80 arch/x86/kernel/fpu/core.c:752) 
[ 1481.383565][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 1481.384625][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.385922][   T32] RIP: 0033:0x7f5c316f8be7
[ 1481.386955][   T32] RSP: 002b:00007ffc354a1f10 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
[ 1481.388659][   T32] RAX: ffffffffffffffda RBX: 0000562309e04d60 RCX: 00007f5c316f8be7
[ 1481.390340][   T32] RDX: 0000000000080000 RSI: 0000562309eb1320 RDI: 00000000ffffff9c
[ 1481.396264][   T32] RBP: 0000562309eb1320 R08: 0000000000000008 R09: 0000000000000001
[ 1481.397989][   T32] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000080000
[ 1481.399627][   T32] R13: 0000562309e04d60 R14: 0000000000000001 R15: 00007ffc354a2320
[ 1481.401415][   T32]  </TASK>
[ 1481.402318][   T32] INFO: task modprobe:90 blocked for more than 983 seconds.
[ 1481.403796][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 1481.405100][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1481.406876][   T32] task:modprobe        state:D stack:0     pid:90    ppid:1      flags:0x00004002
[ 1481.408746][   T32] Call Trace:
[ 1481.409491][   T32]  <TASK>
[ 1481.410226][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 1481.411131][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 1481.412410][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.413686][ T32] ? idr_get_free (arch/x86/include/asm/bitops.h:228 arch/x86/include/asm/bitops.h:240 include/asm-generic/bitops/instrumented-non-atomic.h:142 lib/radix-tree.c:113 lib/radix-tree.c:1518) 
[ 1481.414741][ T32] ? down_write_trylock (kernel/locking/rwsem.c:414) 
[ 1481.415892][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 1481.416818][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 1481.417989][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180) 
[ 1481.419239][ T32] ? down_timeout (kernel/locking/rwsem.c:1108) 
[ 1481.420298][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.421353][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651) 
[ 1481.422530][ T32] ? stack_trace_save (kernel/stacktrace.c:123) 
[ 1481.423640][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574) 
[ 1481.424626][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571) 
[ 1481.425959][ T32] ? kasan_save_stack (mm/kasan/common.c:47) 
[ 1481.427073][ T32] ? kasan_save_stack (mm/kasan/common.c:46) 
[ 1481.428236][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754) 
[ 1481.429310][ T32] ? pcpu_chunk_refresh_hint (mm/percpu-internal.h:114 (discriminator 3) mm/percpu.c:762 (discriminator 3)) 
[ 1481.430476][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044) 
[ 1481.431645][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61) 
[ 1481.432719][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41) 
[ 1481.433765][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.434806][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986) 
[ 1481.436081][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233) 
[ 1481.437226][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451) 
[ 1481.438323][ T32] ? kobject_create_and_add (lib/kobject.c:444) 
[ 1481.439458][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.440515][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153) 
[ 1481.441649][ T32] ? srcu_module_notify (kernel/rcu/srcutree.c:1921 kernel/rcu/srcutree.c:1954) 
[ 1481.442760][ T32] ? tracepoint_module_notify (kernel/tracepoint.c:664 kernel/tracepoint.c:709 kernel/tracepoint.c:701) 
[ 1481.443936][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377) 
[ 1481.444989][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374) 
[ 1481.446267][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343) 
[ 1481.447469][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296) 
[ 1481.448606][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945) 
[ 1481.449662][ T32] load_module (kernel/module/main.c:2965) 
[ 1481.450708][ T32] ? post_relocation (kernel/module/main.c:2829) 
[ 1481.451842][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38) 
[ 1481.452990][ T32] ? __cond_resched (kernel/sched/core.c:8533) 
[ 1481.454080][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099) 
[ 1481.455293][ T32] __do_sys_finit_module (kernel/module/main.c:3099) 
[ 1481.456528][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061) 
[ 1481.457660][ T32] ? randomize_page (mm/util.c:533) 
[ 1481.458766][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445) 
[ 1481.459899][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 1481.460801][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.462053][   T32] RIP: 0033:0x7feb8112a9b9
[ 1481.463109][   T32] RSP: 002b:00007ffd74170a98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 1481.465024][   T32] RAX: ffffffffffffffda RBX: 000055ac05343d30 RCX: 00007feb8112a9b9
[ 1481.466632][   T32] RDX: 0000000000000000 RSI: 000055ac05277260 RDI: 0000000000000003
[ 1481.468414][   T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 000055ac053458b0
[ 1481.470222][   T32] R10: 0000000000000003 R11: 0000000000000246 R12: 000055ac05277260
[ 1481.471905][   T32] R13: 0000000000000000 R14: 000055ac05343cb0 R15: 000055ac05343d30
[ 1481.473600][   T32]  </TASK>
[ 1481.474351][   T32] INFO: task modprobe:91 blocked for more than 983 seconds.
[ 1481.475846][   T32]       Not tainted 6.3.0-12053-ga551138c4b3b #1
[ 1481.477177][   T32] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1481.479050][   T32] task:modprobe        state:D stack:0     pid:91    ppid:1      flags:0x00004002
[ 1481.480969][   T32] Call Trace:
[ 1481.481794][   T32]  <TASK>
[ 1481.482535][ T32] __schedule (kernel/sched/core.c:5343 kernel/sched/core.c:6669) 
[ 1481.483542][ T32] ? io_schedule_timeout (kernel/sched/core.c:6551) 
[ 1481.484706][ T32] ? osq_unlock (kernel/locking/osq_lock.c:22 kernel/locking/osq_lock.c:210) 
[ 1481.485728][ T32] schedule (arch/x86/include/asm/preempt.h:85 (discriminator 1) kernel/sched/core.c:6746 (discriminator 1)) 
[ 1481.486675][ T32] schedule_preempt_disabled (arch/x86/include/asm/preempt.h:80 kernel/sched/core.c:6805) 
[ 1481.487928][ T32] rwsem_down_write_slowpath (arch/x86/include/asm/current.h:41 kernel/locking/rwsem.c:1180) 
[ 1481.489130][ T32] ? down_timeout (kernel/locking/rwsem.c:1108) 
[ 1481.490248][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.491354][ T32] ? __kernfs_new_node (fs/kernfs/dir.c:651) 
[ 1481.492438][ T32] down_write (kernel/locking/rwsem.c:1306 kernel/locking/rwsem.c:1315 kernel/locking/rwsem.c:1574) 
[ 1481.493342][ T32] ? rwsem_down_write_slowpath (kernel/locking/rwsem.c:1571) 
[ 1481.494468][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.495663][ T32] ? notifier_call_chain (kernel/notifier.c:95) 
[ 1481.496756][ T32] ? blocking_notifier_call_chain_robust (kernel/notifier.c:129 kernel/notifier.c:353 kernel/notifier.c:341) 
[ 1481.498045][ T32] ? load_module (include/linux/notifier.h:209 kernel/module/main.c:2764 kernel/module/main.c:2945) 
[ 1481.499131][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099) 
[ 1481.500344][ T32] ? do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 1481.501394][ T32] ? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.502720][ T32] kernfs_add_one (include/linux/kernfs.h:391 fs/kernfs/dir.c:754) 
[ 1481.503801][ T32] kernfs_create_dir_ns (fs/kernfs/dir.c:1044) 
[ 1481.504981][ T32] sysfs_create_dir_ns (fs/sysfs/dir.c:61) 
[ 1481.505997][ T32] ? sysfs_create_mount_point (fs/sysfs/dir.c:41) 
[ 1481.507197][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.509314][ T32] ? __kmalloc_node_track_caller (include/linux/kasan.h:196 mm/slab_common.c:966 mm/slab_common.c:986) 
[ 1481.510619][ T32] kobject_add_internal (lib/kobject.c:65 lib/kobject.c:233) 
[ 1481.511958][ T32] kobject_init_and_add (lib/kobject.c:368 lib/kobject.c:451) 
[ 1481.513079][ T32] ? kobject_create_and_add (lib/kobject.c:444) 
[ 1481.514276][ T32] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:186 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154) 
[ 1481.515339][ T32] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153) 
[ 1481.516463][ T32] ? ddebug_module_notify (lib/dynamic_debug.c:1344 lib/dynamic_debug.c:1336) 
[ 1481.517587][ T32] mod_sysfs_setup (kernel/module/sysfs.c:361 kernel/module/sysfs.c:377) 
[ 1481.518672][ T32] ? module_add_modinfo_attrs (kernel/module/sysfs.c:374) 
[ 1481.519892][ T32] ? atomic_notifier_call_chain (kernel/notifier.c:343) 
[ 1481.521125][ T32] ? klp_module_coming (kernel/livepatch/core.c:1296) 
[ 1481.522271][ T32] ? load_module (kernel/module/main.c:2758 kernel/module/main.c:2945) 
[ 1481.523331][ T32] load_module (kernel/module/main.c:2965) 
[ 1481.524366][ T32] ? post_relocation (kernel/module/main.c:2829) 
[ 1481.525461][ T32] ? __x64_sys_fspick (fs/kernel_read_file.c:38) 
[ 1481.526548][ T32] ? __cond_resched (kernel/sched/core.c:8533) 
[ 1481.527585][ T32] ? __do_sys_finit_module (kernel/module/main.c:3099) 
[ 1481.528682][ T32] __do_sys_finit_module (kernel/module/main.c:3099) 
[ 1481.529764][ T32] ? __ia32_sys_init_module (kernel/module/main.c:3061) 
[ 1481.530866][ T32] ? randomize_page (mm/util.c:533) 
[ 1481.531860][ T32] ? ksys_mmap_pgoff (mm/mmap.c:1445) 
[ 1481.532922][ T32] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) 
[ 1481.533948][ T32] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) 
[ 1481.535161][   T32] RIP: 0033:0x7f1d905f79b9
[ 1481.536136][   T32] RSP: 002b:00007ffc9133bbd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 1481.537935][   T32] RAX: ffffffffffffffda RBX: 00005651b1b1ae20 RCX: 00007f1d905f79b9
[ 1481.539611][   T32] RDX: 0000000000000000 RSI: 00005651b1622260 RDI: 0000000000000003
[ 1481.541313][   T32] RBP: 0000000000060000 R08: 0000000000000000 R09: 00005651b1b1c8b0
[ 1481.543097][   T32] R10: 0000000000000003 R11: 0000000000000246 R12: 00005651b1622260
[ 1481.544859][   T32] R13: 0000000000000000 R14: 00005651b1b1af50 R15: 00005651b1b1ae20
[ 1481.546692][   T32]  </TASK>
[ 1481.547492][   T32] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
BUG: kernel hang in test stage

Kboot worker: lkp-worker18
Elapsed time: 2520

kvm=(
qemu-system-x86_64
-enable-kvm
-cpu SandyBridge
-kernel $kernel
-initrd initrd-vm-meta-147.cgz
-m 16384
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0,hostfwd=tcp::32032-:22
-boot order=nc
-no-reboot
-device i6300esb
-watchdog-action debug
-rtc base=localtime
-serial stdio
-display none
-monitor null
)

append=(
ip=::::vm-meta-147::dhcp
root=/dev/ram0
RESULT_ROOT=/result/boot/1/vm-snb/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3-func/gcc-11/a551138c4b3b9fd7e74a55d6074a013bab6246db/3
BOOT_IMAGE=/pkg/linux/x86_64-rhel-8.3-func/gcc-11/a551138c4b3b9fd7e74a55d6074a013bab6246db/vmlinuz-6.3.0-12053-ga551138c4b3b
branch=linux-review/Dominique-Martinet/fs-split-off-vfs_getdents-function-of-getdents64-syscall/20230510-185542
job=/job-script
user=lkp
ARCH=x86_64
kconfig=x86_64-rhel-8.3-func
commit=a551138c4b3b9fd7e74a55d6074a013bab6246db
initcall_debug
nmi_watchdog=0
vmalloc=256M
initramfs_async=0


To reproduce:

        # build kernel
	cd linux
	cp config-6.3.0-12053-ga551138c4b3b .config
	make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
	make HOSTCC=gcc-11 CC=gcc-11 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
	cd <mod-install-dir>
	find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz


        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email

        # if come across any failure that blocks the test,
        # please remove ~/.lkp and /lkp dir to run from a clean state.



-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests



View attachment "config-6.3.0-12053-ga551138c4b3b" of type "text/plain" (162222 bytes)

View attachment "job-script" of type "text/plain" (5112 bytes)

Download attachment "dmesg.xz" of type "application/x-xz" (30760 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ