linux-kernel - Re: hung_task checking and sys

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120612222912.GB16381@google.com>
Date:	Tue, 12 Jun 2012 15:29:12 -0700
From:	Mandeep Singh Baines <msb@...gle.com>
To:	Daniel Walker <dwalker@...o99.com>
Cc:	fweisbec@...il.com, msb@...omium.org, sshaiju@...sta.com,
	mingo@...e.hu, akpm@...ux-foundation.org,
	linux-kernel@...r.kernel.org
Subject: Re: hung_task checking and sys_sync

Daniel Walker (dwalker@...o99.com) wrote:
> 

Hi Daniel,

> I found this commit which was a while ago,
> 
> commit fb822db465bd9fd4208eef1af4490539b236c54e
> Author: Ingo Molnar <mingo@...e.hu>
> Date:   Wed Aug 20 11:17:40 2008 +0200
> 
>     softlockup: increase hung tasks check from 2 minutes to 8 minutes
>     
>     Andrew says:
>     
>     > Seems that about 100% of the reports we get of this warning triggering
>     > are sys_sync, transaction commit, etc.
>     
>     increase the timeout. If it still triggers for people, we can kill it.
>     
>     Signed-off-by: Ingo Molnar <mingo@...e.hu>
> 
> 
> We're seeing these messages on an older kernel (montavista) but the code areas
> appear similar to current kernels. The issue is that we're doing a file copy
> which takes 10-15minutes, and in the background there is a "df --sync"
> happening (which is calling sys_sync). We end up getting a hung task message
> like below,
> 
> INFO: task df:1778 blocked for more than 120 seconds.
> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> ffffffff81578d40 0000000000000086 ffff8801f6135b00 ffff880269a91800
> ffff880269a91800 ffff8802702be000 ffff8801f602a080 0000000000000000
> ffff8801f602a440 ffffffff8109c166 ffff8801e863de18 0000000000000004
> Call Trace:
>   [<ffffffff8109c166>] ? sync_page+0x0/0x49
>   [<ffffffff81320de2>] ? __schedule+0x3c/0x57
>   [<ffffffff810ea3c7>] ? bdi_sched_wait+0x0/0xe
>   [<ffffffff81320de2>] ? __schedule+0x3c/0x57
>   [<ffffffff81320e0d>] ? schedule+0x10/0x1e
>   [<ffffffff810ea3d0>] ? bdi_sched_wait+0x9/0xe
> 
> There some variation in the stack trace , but always thru bdi_sched_wait().
> 
> 
> These don't seem like valid warnings, since the copy happening is know to take
> a long time.

But the time is not unbounded. You could mask the hung_task_detector for
this case but then you lose the ability to catch bugs in this code path.

The timeout is configurable via /proc/sys/kernel/hung_task_timeout_secs.
Can you bump up the value at boot via sysctl.conf?

> Has there been any commit that disable these messages bdi_sched_wait?
> 

No. There is no mechanism to disable hung_task for a specific code path.
We do skip processes if PF_PROZEN or PF_FROZEN_SKIP is set but that is
really a different situation where the wait is unbounded.

Regards,
Mandeep

> Daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/