lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <70cb224f-a468-40a0-82c7-735c590bdb43@oracle.com>
Date: Thu, 24 Oct 2024 16:49:40 +0100
From: John Garry <john.g.garry@...cle.com>
To: tj@...nel.org, jiangshanlai@...il.com, mingo@...hat.com,
        peterz@...radead.org, juri.lelli@...hat.com
Cc: jack@...e.cz, david@...morbit.com, akpm@...ux-foundation.org,
        linux-kernel@...r.kernel.org
Subject: workqueue lockup debug

Hi workqueue and scheduler maintainers,

As reported in 
https://lore.kernel.org/linux-fsdevel/df9db1ce-17d9-49f1-ab6d-7ed9a4f1f9c0@oracle.com/T/#m506b9edb1340cdddd87c6d14d20222ca8d7e8796, 
I am experiencing a workqueue lockup for v6.12-rcX.

At the point it occurs, the system becomes unresponsive and I cannot 
bring it back to life.

Enabling /proc/sys/kernel/softlockup_all_cpu_backtrace does not give 
anything extra in the way of debug. All I get is something like this:

Message from syslogd@...rry-atomic-write-exp-e4-8-instance-20231214-1221 
at Oct 24 15:34:02 ...
  kernel:watchdog: BUG: soft lockup - CPU#29 stuck for 22s! [mysqld:14352]

Message from syslogd@...rry-atomic-write-exp-e4-8-instance-20231214-1221 
at Oct 24 15:34:02 ...
  kernel:BUG: workqueue lockup - pool cpus=4 node=0 flags=0x0 nice=0 
stuck for 30s!

Message from syslogd@...rry-atomic-write-exp-e4-8-instance-20231214-1221 
at Oct 24 15:34:02 ...
  kernel:BUG: workqueue lockup - pool cpus=31 node=0 flags=0x0 nice=0 
stuck for 49s!
^C

Can you advise on a robust method to get some debug from this system?

Maybe this is a scheduler issue, as Dave mentioned in that same thread.

Thanks,
John

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ