linux-kernel - Re: [peterz-queue:sched/core] [sched/fair] 420356c350: WARNING:at_kernel/sched/core.c:#__might

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <Zs2e988qjFLpAnxX@chenyu5-mobl2>
Date: Tue, 27 Aug 2024 17:40:07 +0800
From: Chen Yu <yu.c.chen@...el.com>
To: Peter Zijlstra <peterz@...radead.org>
CC: Oliver Sang <oliver.sang@...el.com>, <oe-lkp@...ts.linux.dev>,
	<lkp@...el.com>, <linux-kernel@...r.kernel.org>, <aubrey.li@...ux.intel.com>
Subject: Re: [peterz-queue:sched/core] [sched/fair]  420356c350:
 WARNING:at_kernel/sched/core.c:#__might_sleep

On 2024-08-26 at 16:25:56 +0800, Chen Yu wrote:
> On 2024-08-22 at 17:49:23 +0200, Peter Zijlstra wrote:
> > On Mon, Aug 19, 2024 at 12:44:39PM +0800, Chen Yu wrote:
> > > On 2024-08-17 at 11:33:29 +0200, Peter Zijlstra wrote:
> > > > On Fri, Aug 16, 2024 at 05:15:12PM +0800, kernel test robot wrote:
> > > > > kernel test robot noticed "WARNING:at_kernel/sched/core.c:#__might_sleep" on:
> > > > > 
> > > > > commit: 420356c3504091f0f6021974389df7c58f365dad ("sched/fair: Implement delayed dequeue")
> > > > > https://git.kernel.org/cgit/linux/kernel/git/peterz/queue.git sched/core
> > > > 
> > > > > [   86.252370][  T674] ------------[ cut here ]------------
> > > > > [ 86.252945][ T674] do not call blocking ops when !TASK_RUNNING; state=1 set at kthread_worker_fn (kernel/kthread.c:?) 
> > > > > [ 86.254001][ T674] WARNING: CPU: 1 PID: 674 at kernel/sched/core.c:8469 __might_sleep (kernel/sched/core.c:8465) 
> > > > 
> > > > > [ 86.283398][ T674] ? handle_bug (arch/x86/kernel/traps.c:239) 
> > > > > [ 86.283995][ T674] ? exc_invalid_op (arch/x86/kernel/traps.c:260) 
> > > > > [ 86.284787][ T674] ? asm_exc_invalid_op (arch/x86/include/asm/idtentry.h:621) 
> > > > > [ 86.285682][ T674] ? __might_sleep (kernel/sched/core.c:8465) 
> > > > > [ 86.286380][ T674] ? __might_sleep (kernel/sched/core.c:8465) 
> > > > > [ 86.287116][ T674] kthread_worker_fn (include/linux/kernel.h:73 include/linux/freezer.h:53 kernel/kthread.c:851) 
> > > > > [ 86.287701][ T674] ? kthread_worker_fn (kernel/kthread.c:?) 
> > > > > [ 86.288138][ T674] kthread (kernel/kthread.c:391) 
> > > > > [ 86.288482][ T674] ? __cfi_kthread_worker_fn (kernel/kthread.c:803) 
> > > > > [ 86.288951][ T674] ? __cfi_kthread (kernel/kthread.c:342) 
> > > > > [ 86.289560][ T674] ret_from_fork (arch/x86/kernel/process.c:153) 
> > > > > [ 86.290162][ T674] ? __cfi_kthread (kernel/kthread.c:342) 
> > > > > [ 86.291465][ T674] ret_from_fork_asm (arch/x86/entry/entry_64.S:254) 
> > > > 
> > > > AFAICT this is a pre-existing issue. Notably that all transcribes to:
> > > > 
> > > > kthread_worker_fn()
> > > >   ...
> > > > repeat:
> > > >   set_current_state(TASK_INTERRUPTIBLE);
> > > >   ...
> > > >   if (work) { // false
> > > >     __set_current_state(TASK_RUNNING);
> > > >     ...
> > > >   } else if (!freezing(current)) // false -- we are freezing
> > > >     schedule();
> > > > 
> > > >   // so state really is still TASK_INTERRUPTIBLE here
> > > >   try_to_freeze()
> > > >     might_sleep() <--- boom, per the above.
> > > >
> > > 
> > > Would the following fix make sense?
> > 
> > Yeah, that looks fine. Could you write it up as a proper patch please?
> >
> 
> Yes, it should be a race condition in theory and I've sent a patch here:
> https://lore.kernel.org/lkml/20240819141551.111610-1-yu.c.chen@intel.com/
> And Andrew has given some comments on it.
> 
> However, after I did some further investigation, this warning seems to
> not be directly related to task freeze, but has connection with the
> delay dequeue. I'm planning to add debug patch and investigate the
> symptom in 0day's environment, will send the finding later.
>

The root cause is replied to the delayed dequeue patch set here:
https://lore.kernel.org/lkml/Zs2ZoAcUsZMX2B%2FI@chenyu5-mobl2/

And since the race condition mentioned previously is real
(although not the root cause of the warning reported in this thread),
I'll send a v2 patch to get that fixed.

thanks,
Chenyu