linux-kernel - Re: Bug 218665 - nohz_full=0 prevents kernel from booting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <enqg6mcuhvff7gujjbapdiclicl3z6f2vnggcsg65pexipyr3o@4men5fhyt3vb>
Date: Sun, 7 Apr 2024 15:52:52 -0700
From: Bjorn Andersson <andersson@...nel.org>
To: Linux regressions mailing list <regressions@...ts.linux.dev>
Cc: Tejun Heo <tj@...nel.org>, LKML <linux-kernel@...r.kernel.org>
Subject: Re: Bug 218665 - nohz_full=0 prevents kernel from booting

On Tue, Apr 02, 2024 at 10:17:16AM +0200, Linux regression tracking (Thorsten Leemhuis) wrote:
> Hi, Thorsten here, the Linux kernel's regression tracker.
> 
> I noticed a regression report in bugzilla.kernel.org. As many (most?)
> kernel developers don't keep an eye on it, I decided to forward it by mail.
> 
> Tejun, apparently it's cause by a change of yours.
> 
> Note, you have to use bugzilla to reach the reporter, as I sadly[1] can
> not CCed them in mails like this.
> 
> Quoting from https://bugzilla.kernel.org/show_bug.cgi?id=218665 :
> 
> > booting the current kernel (6.9.0-rc1, master/712e1425) on x86_64
> > with nohz_full=0 cause a page fault and prevents the kernel from
> > booting.
> > 
> > Steps to reproduce:
> > - make defconfig
> > - set CONFIG_NO_HZ_FULL=y
> > - set CONFIG_SUSPEND=n and CONFIG_HIBERNATION=n (to get CONFIG_PM_SLEEP_SMP=n)
> > - make
> > - qemu-system-x86_64 -nographic -cpu qemu64 -smp cores=2 -m 1024 -kernel arch/x86/boot/bzImage -append "earlyprintk=ttyS0 console=ttyS0 root=/dev/dummy rootwait nohz_full=0"
> > 
> > I have attached the output of a failed nohz_full=0 boot as
> > nohz_full_0.txt and - for reference - the output of a nohz_full=1
> > boot as nohz_full_1.txt.
> > 
> > Interestingly enough, using the deprecated isolcpus parameter to
> > enable NO_HZ for cpu0 works. I've attached the output as
> > isolcpus_nohz_0.txt.
> > 
> > Bisecting showed 5797b1c18919cd9c289ded7954383e499f729ce0 as first bad commit.
> 
> See the ticket for more details.
> 
> Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
> --
> Everything you wanna know about Linux kernel regression tracking:
> https://linux-regtracking.leemhuis.info/about/#tldr
> If I did something stupid, please tell me, as explained on that page.
> 
> [1] because bugzilla.kernel.org tells users upon registration their
> "email address will never be displayed to logged out users"
> 
> #regzbot introduced: 5797b1c18919cd9c289ded7954383e499f729ce0
> #regzbot from: Friedrich Oslage
> #regzbot duplicate https://bugzilla.kernel.org/show_bug.cgi?id=218665
> #regzbot title: workqueue: nohz_full=0 prevents booting
> #regzbot ignore-activity

In addition to this report, I have finally bisected another regression
to the same commit:

I start neovim, send SIGSTOP (i.e. ^Z) to it, start another neovim
instance and upon sending SIGSTOP to that instance all of userspace
locks up - 100% reproducible.

The kernel seems to continue to operate, and tapping the power button
dislodge the lockup and I get a clean shutdown.

This is seen on multiple Arm64 (Qualcomm) machines with upstream
defconfig since commit '5797b1c18919 ("workqueue: Implement system-wide
nr_active enforcement for unbound workqueues")'.

Regards,
Bjorn