linux-kernel - Re: BUG: soft lockup in 2.6.25.5

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20080919015502.4183933d.akpm@linux-foundation.org>
Date:	Fri, 19 Sep 2008 01:55:02 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Brett Pemberton <brett@...c.org>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: BUG: soft lockup in 2.6.25.5

On Fri, 19 Sep 2008 13:49:16 +1000 Brett Pemberton <brett@...c.org> wrote:

> I'm getting about 3-5 machines in a cluster of 95 hanging with 
> 
> BUG: soft lockup - CPU#7 stuck for 61s! [pdflush:321]
> 
> per week.  Nothing in common each time, different users running
> different jobs on different nodes.
> 
> The most recent is at the end of this email, .config is attached.
> 
> Googling is scary.  Many people reporting these, but never any response.
> It's happening on enough separate nodes that I can't believe it's
> hardware, although they are identical machines:
> 
> - 2x Quad-Core AMD Opteron(tm) Processor 2356
> - 32gb ram
> - 4 x sata drives
> 
> Running CentOS 5.2 with a kernel.org kernel
> Has been happening with a variety of kernels from 2.6.25 - present.

Yes, it's a false positive.  With a lot of memory and a random-access
or lot-of-files writing behaviour, it can take tremendous amounts of
time to get everything stored on the disk.

Not sure what to do about it, really.  Perhaps touch the softlockup
detector somewhere in the writeback code.

> I'd love any advice on where to turn to next and what avenues to pursue.

Set /proc/sys/kernel/softlockup_thresh to zero to shut it up :(

hm, there's a bug...

--- a/Documentation/sysctl/kernel.txt~a
+++ a/Documentation/sysctl/kernel.txt
@@ -351,9 +351,10 @@ kernel.  This value defaults to SHMMAX.
 
 softlockup_thresh:
 
-This value can be used to lower the softlockup tolerance
-threshold. The default threshold is 10s.  If a cpu is locked up
-for 10s, the kernel complains.  Valid values are 1-60s.
+This value can be used to lower the softlockup tolerance threshold.  The
+default threshold is 60 seconds.  If a cpu is locked up for 60 seconds, the
+kernel complains.  Valid values are 1-60 seconds.  Setting this tunable to
+zero will disable the softlockup detection altogether.
 
 ==============================================================
 
_

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/