| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Thu, 1 Feb 2024 14:59:00 +0100
From: "Rafael J. Wysocki" <rafael@...nel.org>
To: Vimal Kumar <vimal.kumar32@...il.com>
Cc: kernel test robot <oliver.sang@...el.com>, oe-lkp@...ts.linux.dev, lkp@...el.com,
Chinmoy Ghosh <chinmoyghosh2001@...il.com>, Mintu Patel <mintupatel89@...il.com>,
Vishal Badole <badolevishal1116@...il.com>, linux-pm@...r.kernel.org, rui.zhang@...el.com,
yu.c.chen@...el.com, "Rafael J. Wysocki" <rafael@...nel.org>, Len Brown <len.brown@...el.com>,
Pavel Machek <pavel@....cz>, Greg Kroah-Hartman <gregkh@...uxfoundation.org>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3] PM / sleep: Mechanism to find source aborting kernel
suspend transition
On Thu, Feb 1, 2024 at 5:44 AM Vimal Kumar <vimal.kumar32@...il.com> wrote:
>
> On Mon, Jan 29, 2024 at 10:08:32PM +0800, kernel test robot wrote:
> >
> >
> > Hello,
> >
> > kernel test robot noticed "BUG:sleeping_function_called_from_invalid_context_at_include/linux/sched/mm.h" on:
> >
> > commit: e2d228456d500f3b00ad746b353fa85eee235b7a ("[PATCH v3] PM / sleep: Mechanism to find source aborting kernel suspend transition")
> > url: https://github.com/intel-lab-lkp/linux/commits/Vimal-Kumar/PM-sleep-Mechanism-to-find-source-aborting-kernel-suspend-transition/20240109-210519
> > base: https://git.kernel.org/cgit/linux/kernel/git/rafael/linux-pm.git linux-next
> > patch link: https://lore.kernel.org/all/20240109130210.31938-1-vimal.kumar32@gmail.com/
> > patch subject: [PATCH v3] PM / sleep: Mechanism to find source aborting kernel suspend transition
> >
> > in testcase: suspend-stress
> > version:
> > with following parameters:
> >
> > mode: freeze
> > iterations: 10
> >
> >
> >
> > compiler: gcc-12
> > test machine: 4 threads (Broadwell) with 8G memory
> >
> > (please refer to attached dmesg/kmsg for entire log/backtrace)
> >
> >
> >
> > If you fix the issue in a separate patch/commit (i.e. not just a new version of
> > the same patch/commit), kindly add following tags
> > | Reported-by: kernel test robot <oliver.sang@...el.com>
> > | Closes: https://lore.kernel.org/oe-lkp/202401292109.f115a688-oliver.sang@intel.com
> >
> >
> > kern :err : [ 97.935754] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:306
> > kern :err : [ 97.935886] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/1
> > kern :err : [ 97.936000] preempt_count: 10002, expected: 0
> > kern :err : [ 97.936071] RCU nest depth: 0, expected: 0
> > kern :warn : [ 97.936138] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.7.0-rc8-00169-ge2d228456d50 #1
> > kern :warn : [ 97.936255] Hardware name: /NUC5i3RYB, BIOS RYBDWi3586A.0363.2017.0316.1028 03/16/2017
> > kern :warn : [ 97.936371] Call Trace:
> > kern :warn : [ 97.936419] <IRQ>
> > kern :warn : [ 97.936461] dump_stack_lvl (lib/dump_stack.c:107 (discriminator 1))
> > kern :warn : [ 97.936529] __might_resched (kernel/sched/core.c:10152)
> > kern :warn : [ 97.936598] ? preempt_notifier_dec (kernel/sched/core.c:10106)
> > kern :warn : [ 97.936672] __kmem_cache_alloc_node (include/linux/kernel.h:107 include/linux/sched/mm.h:306 mm/slab.h:710 mm/slub.c:3460 mm/slub.c:3517)
> > kern :warn : [ 97.936750] ? kasprintf (lib/kasprintf.c:54)
> > kern :warn : [ 97.936812] ? pointer (lib/vsprintf.c:2755)
> > kern :warn : [ 97.936875] ? kasprintf (lib/kasprintf.c:54)
> > kern :warn : [ 97.936936] __kmalloc_node_track_caller (include/linux/kasan.h:198 mm/slab_common.c:1007 mm/slab_common.c:1027)
> > kern :warn : [ 97.937017] kvasprintf (lib/kasprintf.c:25)
> > kern :warn : [ 97.937079] ? bust_spinlocks (lib/kasprintf.c:16)
> > kern :warn : [ 97.937146] ? enqueue_hrtimer (kernel/time/hrtimer.c:1095 (discriminator 3))
> > kern :warn : [ 97.937214] ? tick_nohz_highres_handler (kernel/time/tick-sched.c:1530)
> > kern :warn : [ 97.937295] kasprintf (lib/kasprintf.c:54)
> > kern :warn : [ 97.937354] ? kvasprintf_const (lib/kasprintf.c:54)
> > kern :warn : [ 97.937426] ? _raw_spin_lock_irqsave (arch/x86/include/asm/atomic.h:115 include/linux/atomic/atomic-arch-fallback.h:2164 include/linux/atomic/atomic-instrumented.h:1296 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:187 include/linux/spinlock_api_smp.h:111 kernel/locking/spinlock.c:162)
> > kern :warn : [ 97.937501] ? _raw_read_unlock_irqrestore (kernel/locking/spinlock.c:161)
> > kern :warn : [ 97.937581] ? _raw_spin_lock (arch/x86/include/asm/atomic.h:115 include/linux/atomic/atomic-arch-fallback.h:2164 include/linux/atomic/atomic-instrumented.h:1296 include/asm-generic/qspinlock.h:111 include/linux/spinlock.h:187 include/linux/spinlock_api_smp.h:134 kernel/locking/spinlock.c:154)
> > kern :warn : [ 97.937648] ? _raw_write_lock_irq (kernel/locking/spinlock.c:153)
> > kern :warn : [ 97.937719] ? irq_pm_check_wakeup (kernel/irq/pm.c:24)
> > kern :warn : [ 97.937793] pm_system_wakeup (drivers/base/power/wakeupc:981)
> > kern :warn : [ 97.937862] irq_pm_check_wakeup (kernel/irq/pm.c:24)
> > kern :warn : [ 97.937933] handle_fasteoi_irq (kernel/irq/chip.c:518 kernel/irq/chip.c:502 kernel/irq/chip.c:698)
> > kern :warn : [ 97.938006] __common_interrupt (arch/x86/kernel/irq.c:271 (discriminator 22))
> > kern :warn : [ 97.938078] common_interrupt (arch/x86/kernel/irq.c:247 (discriminator 14))
> > kern :warn : [ 97.938145] </IRQ>
> > kern :warn : [ 97.938187] <TASK>
> > kern :warn : [ 97.938230] asm_common_interrupt (arch/x86/include/asm/idtentry.h:640)
> > kern :warn : [ 97.938301] RIP: 0010:cpuidle_enter_s2idle (arch/x86/include/asm/irqflags.h:78 drivers/cpuidle/cpuidle.c:199)
> > kern :warn : [ 97.938384] Code: d9 75 83 85 d2 7f 11 48 83 c4 20 89 d0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 4c 89 c6 48 89 cf 89 14 24 e8 fb 64 9b 00 fb 8b 14 24 <eb> db 31 d2 eb d7 48 89 0c 24 48 89 74 24 08 e8 03 51 9f fe 4c 8b
> > All code
> > ========
> > 0: d9 75 83 fnstenv -0x7d(%rbp)
> > 3: 85 d2 test %edx,%edx
> > 5: 7f 11 jg 0x18
> > 7: 48 83 c4 20 add $0x20,%rsp
> > b: 89 d0 mov %edx,%eax
> > d: 5b pop %rbx
> > e: 5d pop %rbp
> > f: 41 5c pop %r12
> > 11: 41 5d pop %r13
> > 13: 41 5e pop %r14
> > 15: 41 5f pop %r15
> > 17: c3 retq
> > 18: 4c 89 c6 mov %r8,%rsi
> > 1b: 48 89 cf mov %rcx,%rdi
> > 1e: 89 14 24 mov %edx,(%rsp)
> > 21: e8 fb 64 9b 00 callq 0x9b6521
> > 26: fb sti
> > 27: 8b 14 24 mov (%rsp),%edx
> > 2a:* eb db jmp 0x7 <-- trapping instruction
> > 2c: 31 d2 xor %edx,%edx
> > 2e: eb d7 jmp 0x7
> > 30: 48 89 0c 24 mov %rcx,(%rsp)
> > 34: 48 89 74 24 08 mov %rsi,0x8(%rsp)
> > 39: e8 03 51 9f fe callq 0xfffffffffe9f5141
> > 3e: 4c rex.WR
> > 3f: 8b .byte 0x8b
> >
> > Code starting with the faulting instruction
> > ===========================================
> > 0: eb db jmp 0xffffffffffffffdd
> > 2: 31 d2 xor %edx,%edx
> > 4: eb d7 jmp 0xffffffffffffffdd
> > 6: 48 89 0c 24 mov %rcx,(%rsp)
> > a: 48 89 74 24 08 mov %rsi,0x8(%rsp)
> > f: e8 03 51 9f fe callq 0xfffffffffe9f5117
> > 14: 4c rex.WR
> > 15: 8b .byte 0x8b
> > kern :warn : [ 97.938632] RSP: 0018:ffffc90000177db8 EFLAGS: 00000286
> > kern :warn : [ 97.938717] RAX: 00000000037a402c RBX: 0000000000000009 RCX: 0000000000000000
> > kern :warn : [ 97.938823] RDX: 0000000000000008 RSI: 0000000937e23ad2 RDI: 0000000000000000
> > kern :warn : [ 97.938927] RBP: ffffffff84f7f680 R08: 0000000000000000 R09: ffffed1021ad5530
> > kern :warn : [ 97.939032] R10: ffff88810d6aa987 R11: 0000000000000000 R12: ffff88820decca10
> > kern :warn : [ 97.939137] R13: dffffc0000000000 R14: 000000000027ac40 R15: 000000000027ac40
> > kern :warn : [ 97.939244] ? cpuidle_enter_s2idle (arch/x86/include/asm/irqflags.h:42 arch/x86/include/asm/irqflags.h:77 drivers/cpuidle/cpuidle.c:199)
> > kern :warn : [ 97.939319] cpuidle_idle_call (kernel/sched/idle.c:190)
> > kern :warn : [ 97.939389] ? arch_cpu_idle_exit+0x40/0x40
> > kern :warn : [ 97.939459] ? mark_tsc_async_resets (arch/x86/kernel/tsc_sync.c:51)
> > kern :warn : [ 97.939536] do_idle (kernel/sched/idle.c:282)
> > kern :warn : [ 97.939594] cpu_startup_entry (kernel/sched/idle.c:379 (discriminator 1))
> > kern :warn : [ 97.939662] start_secondary (arch/x86/kernel/smpboot.c:224 arch/x86/kernel/smpboot.c:304)
> > kern :warn : [ 97.939731] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:254)
> > kern :warn : [ 97.939808] ? soft_restart_cpu (arch/x86/kernel/head_64S:485)
> > kern :warn : [ 97.939878] secondary_startup_64_no_verify (arch/x86/kernel/head_64.S:448)
> > kern :warn : [ 97.939963] </TASK>
> > kern :info : [ 97.948803] bdw_uncore 0000:00:00.0: PM: calling pci_pm_resume_noirq+0x0/0x390 @ 85, parent: 00
> > kern :info : [ 97.948993] bdw_uncore 0000:00:00.0: PM: pci_pm_resume_noirq+0x0/0x390 returned 0 after 17 usecs
> >
> >
> >
> > The kernel config and materials to reproduce are available at:
> > https://download.01.org/0day-ci/archive/20240129/202401292109.f115a688-oliver.sang@intel.com
> >
> >
> >
> > --
> > 0-DAY CI Kernel Test Service
> > https://github.com/intel/lkp-tests/wiki
> >
>
> Hi Greg,
>
> This is an example where "pm_system_wakeup" can be called from atomic context. It also gives an actual instance where we can utilise this feature to find the source.
>
> The bug reported due to "GFP_KERNEL" flag, but it will be applicable for using "mutex_lock" as well.
> I will change to ATOMIC flag in kmalloc, as well as use raw_spin_lock* instead of mutex_lock.
Maybe instead of trying to add bandaids, you could work on making the
wakeup sources infrastructure suitable for addressing your use case?
Powered by blists - more mailing lists