linux-kernel - Re: [linus:master] [x86/cpu] f388f60ca9: BUG:soft_lockup-CPU##stuck

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <f1ccb8b4-bbe2-42bc-bb86-c2bf3f9c557d@app.fastmail.com>
Date: Thu, 24 Apr 2025 09:59:38 +0200
From: "Arnd Bergmann" <arnd@...db.de>
To: "kernel test robot" <oliver.sang@...el.com>
Cc: oe-lkp@...ts.linux.dev, "kernel test robot" <lkp@...el.com>,
 linux-kernel@...r.kernel.org, "Ingo Molnar" <mingo@...nel.org>,
 "Linus Torvalds" <torvalds@...ux-foundation.org>,
 "John Stultz" <jstultz@...gle.com>, "Thomas Gleixner" <tglx@...utronix.de>,
 "Stephen Boyd" <sboyd@...nel.org>, "Borislav Petkov" <bp@...en8.de>,
 "Dave Hansen" <dave.hansen@...ux.intel.com>, x86@...nel.org,
 "H. Peter Anvin" <hpa@...or.com>
Subject: Re: [linus:master] [x86/cpu]  f388f60ca9:
 BUG:soft_lockup-CPU##stuck_for#s![swapper:#]

On Thu, Apr 24, 2025, at 04:12, Oliver Sang wrote:
> On Tue, Apr 22, 2025 at 12:16:33PM +0200, Arnd Bergmann wrote:

Cc:  x86 and timekeeping maintainers, see
https://lore.kernel.org/lkml/202504211553.3ba9400-lkp@intel.com/
for the thread so far.

>> > [ 721.016779][ C0] hardirqs last disabled at (159506): 
>> > sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1049) 
>> > [ 721.016779][ C0] softirqs last enabled at (159174): handle_softirqs 
>> > (kernel/softirq.c:408 kernel/softirq.c:589) 
>> > [ 721.016779][ C0] softirqs last disabled at (159159): __do_softirq 
>> > (kernel/softirq.c:596) 
>> > [  721.016779][    C0] CPU: 0 UID: 0 PID: 1 Comm: swapper Not tainted 
>> > 6.14.0-rc3-00037-gf388f60ca904 #1
>> > [  721.016779][    C0] Hardware name: QEMU Standard PC (i440FX + PIIX, 
>> > 1996), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
>> > [ 721.016779][ C0] EIP: timekeeping_notify 
>> > (kernel/time/timekeeping.c:1522) 
>> 
>> Timekeeping code could be related, I see that CONFIG_X86_TSC
>> is disabled for i486SX configurations, so even if a TSC is present
>> in the emulated machine, it is not being used to measure time
>> accurately.
>> 
>> > -CONFIG_X86_CMPXCHG64=y
>> 
>> This could be another issue, if there is code that relies on
>> the cx8/cmpxchg8b feature to be used. Since this is a non-SMP
>> kernel, this is less likely to be the cause of the problem.
>
> thanks a lot for all these details!
>
>> 
>> Can you try what happens when you enable the two options, either
>> by changing CONFIG_M486SX to CONFIG_M586TSC, or with a patch
>> like the one below? Note that CONFIG_X86_CMPXCHG64 recently
>> got renamed to CONFIG_X86_CX8, but they are the exact same thing.
>
> I applied your patch directly upon f388f60ca9 (change for X86_CMPXCHG64
> instead of X86_CX8 as you metnioned), commit id is
> c1f7ef63239411313163a7b1bff654236f48351c
>
...
> by running same tests, now it backs to the clean status like
> fc2d5cbe541032e7 (parent of f388f60ca9)

Thanks for confirming. So a 486-targeted kernel still passes
your tests on modern hardware if we force TSC and CX8 to
be enabled, but the boot fails if the options are turned
off in Kconfig (though available in emulated hardware).

To be completely sure, you could re-run the same test with
just one of these enabled, but I'm rather sure that the TSC
is the root cause. I tried reproducing the problem locally
with your .config on a qemu/tcg emulation running on an
arm64 host, but this seems to run fine, including the
rcutorture tests.

Comparing my results with your log file, I see that your
crash happens while changing the clocksource:

Your dmesg:
[   92.548514][    T1] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[  721.016745][    C0] watchdog: BUG: soft lockup - CPU#0 stuck for 626s! [swapper:1]

My dmesg:
[    1.154511][    T1] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[    1.157896][    T1] clocksource: Switched to clocksource tsc-early

There are also clearly some differences between TCG and KVM in
the handling of TSC, e.g. I get this warning from qemu itself
for the SandyBridge CPU:

qemu-system-i386: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24]

I tried a few other variations, including KVM on an x86 laptop
(using kvmclock or tsc-early clocksource), but none of them failed
the way yours did.

      Arnd