linux-kernel - [BUG] 8e274732115f ("printk: extend console

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20220610205038.GA3050413@paulmck-ThinkPad-P17-Gen-1>
Date:   Fri, 10 Jun 2022 13:50:38 -0700
From:   "Paul E. McKenney" <paulmck@...nel.org>
To:     john.ogness@...utronix.de
Cc:     linux-kernel@...r.kernel.org, frederic@...nel.org, pmladek@...e.com
Subject: [BUG] 8e274732115f ("printk: extend console_lock for per-console
 locking")

Hello, John,

I have started getting rcutorture shutdown-time hangs when running
against recent mainline, and bisected back to the 8e274732115f ("printk:
extend console_lock for per-console locking") commit.  These hangs go
away (or at least their probability drops dramatically) if I build with
CONFIG_PREEMPTION=n -and- CONFIG_NO_HZ=y (not n!), at least assuming
that I also boot with "nohz_full=0-N".

Attempts to debug using rcutorture's "--gdb" argument result in
"[Inferior 1 (process 1) exited normally]", but with the same truncated
console-log output as when running without "--gdb".  This suggests
that the kernel really did run to completion and halt as expected,
but that the shutdown-time printk() output was lost.  Furthermore, if I
use the gdb "hbreak" command to set a breakpoint at kernel_power_off(),
it really does hit that breakpoint.  This would not happen in the case
of a kernel hang.

So, given that I can hit that breakpoint, what should I ask gdb to
show me?

Alternatively, this reproduces on a variety of x86 platforms, so you
should be able reproduce it as follows [1]:

	git checkout v5.19-rc1
	tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 2 --configs "TREE01" --gdb "CONFIG_DEBUG_INFO_NONE=n CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y" --trust-make

This builds a kernel, boots it, and then tells you how to launch gdb
(presumably in some other window).  Once you give launch gdb and give
it the suggested commands, the kernel runs for two minutes under qemu,
then shuts down.  I used the following gdb commands to set the breakpoint
and run the kernel:

	target remote :1234  # suggested by the rcutorture script
	hbreak kernel_power_off  # added by me
	continue  # suggested by the rcutorture script

Or leave out the "gdb" if you prefer some other debugging approach.

Thoughts?

							Thanx, Paul

------------------------------------------------------------------------

[1]	The --kconfig argument is needed due to CONFIG_DEBUG_INFO Kconfig
	changes.  You can instead pull in -rcu commit f9993c8413d1
	("torture: Adjust to again produce debugging information"),
	which allows you to omit that --kconfig argument.