linux-kernel - Re: High load average on idle machine running 2.6.32

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 18 Dec 2009 15:12:31 +0100
From:	Peter Zijlstra <peterz@...radead.org>
To:	Andrea Suisani <sickpig@...nioni.net>
Cc:	James Pearson <james-p@...ing-picture.com>,
	linux-kernel@...r.kernel.org, Russell King <linux@....linux.org.uk>
Subject: Re: High load average on idle machine running 2.6.32

On Fri, 2009-12-18 at 14:43 +0100, Andrea Suisani wrote:

> >>> Strangely, when I run 'iftop' (from 
> >>> http://www.ex-parrot.com/pdw/iftop/) using the 2.6.32 kernel, the 
> >>> load average drops to below 0.5  - stop running iftop, and the load 
> >>> average climbs again ...

This is the thing that puzzles me most..

> >> Also, if I 'hot-unplug' CPUs 1 to 7, the load average drops to 0 - 
> >> when I re-enable theses CPUs, the load average climbs.

Very curious too

> >> I guess this is a problem with my particular config - or maybe because 
> >> I'm using NFS-root (the root file system is readonly), or using a 
> >> non-module kernel?

Russell, you grumbled something like this on IRC, are you too using
NFS-root?

> > I gave 'git bisect' a go - which appears to suggest that my problem 
> > started at:
> > 
> > % git bisect bad
> > d7c33c4930f569caf6b2ece597432853c4151a45 is first bad commit
> > commit d7c33c4930f569caf6b2ece597432853c4151a45
> > Author: Peter Zijlstra <a.p.zijlstra@...llo.nl>
> > Date:   Fri Sep 11 12:45:38 2009 +0200
> > 
> >     sched: Fix task affinity for select_task_rq_fair
> > 
> >     While merging select_task_rq_fair() and sched_balance_self() I made
> >     a mistake that leads to testing the wrong task affinty.
> > 
> >     Signed-off-by: Peter Zijlstra <a.p.zijlstra@...llo.nl>
> >     LKML-Reference: <new-submission>
> >     Signed-off-by: Ingo Molnar <mingo@...e.hu>
> > 
> > :040000 040000 3d7aa3e193c7faf9c7ebbb1443c6f63269d86d04 
> > 9cfb647eb5d80f156fd8a495da68f765c3fdd772 M      kernel

> > So I guess, it is not just one patch that has caused the issue I'm 
> > seeing, which I guess is to be expected as the above patch was part of 
> > the 'scheduler updates for v2.6.32' patch set 

Right, so the thing that seems most likely to cause such funnies is the
introduction of TASK_WAKING state in .32, during development we had a
brief period where we saw what you described, but I haven't seen it
after:

commit eb24073bc1fe3e569a855cf38d529fb650c35524
Author: Ingo Molnar <mingo@...e.hu>
Date:   Wed Sep 16 21:09:13 2009 +0200

    sched: Fix TASK_WAKING & loadaverage breakage

> > I guess as no one else has reported this issue - it must be something to 
> > do with my set up - could using NFS-root affect how the load average is 
> > calculated?

So the thing that contributes to load is TASK_UNINTERRUPTIBLE sleeps
(and !PF_FREEZING) as tested by task_contributes_to_load().

Are you seeing a matching number of tasks being stuck in 'D' state when
the load is high? If so, how are these tasks affected by iftop/hotplug?



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/