linux-kernel - Re: [PATCH V2] writeback: fix hung

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <x49vcinhuv1.fsf@segfault.boston.devel.redhat.com>
Date:	Tue, 19 Jun 2012 17:09:22 -0400
From:	Jeff Moyer <jmoyer@...hat.com>
To:	Dave Chinner <david@...morbit.com>
Cc:	Fengguang Wu <fengguang.wu@...el.com>,
	Wanpeng Li <liwp.linux@...il.com>,
	Alexander Viro <viro@...iv.linux.org.uk>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	Gavin Shan <shangw@...ux.vnet.ibm.com>
Subject: Re: [PATCH V2] writeback: fix hung_task alarm when sync block

Dave Chinner <david@...morbit.com> writes:

> On Tue, Jun 19, 2012 at 04:14:16PM -0400, Jeff Moyer wrote:
>> Fengguang Wu <fengguang.wu@...el.com> writes:
>> 
>> > Good idea! Yes we can do some estimation and adaptively extend the
>> > hang timeout for the current writeback_inodes_sb_nr()/sync_inodes_sb()
>> > call.
>> >
>> > Note that it's not going to reliably get rid of false warnings due to
>> > estimation errors, which could be pretty large and unavoidable on
>> > change of workload. But still, it would be a net improvement and
>> > perhaps enough to get rid of most false warnings, while still being
>> > able to catch livelock or other kind of task hang.
>> 
>> Hi, Fengguang,
>> 
>> I didn't see a patch from you for this, so I went ahead and threw
>> something together.  Let me know what you think of it.  I wasn't sure
>> how to estimate the total I/O that will be issued for syncing out an
>> entire superblock, though, so I didn't do that part.
>
> As I said to the original patch - having a hang check timeout on a
> system that is overloaded w.r.t. IO is an important piece of
> information when it comes to debugging problems. Often the hangcheck
> timer is the first piece of information that we will get that
> indicates a problem somewhere in a production system.

So, you believe that we should always check at 2 minute intervals (or
whatever is configured), even if we know there is more than that much
I/O queued?  In case there is any confusion, here, the patch I posted
ensured that we would eventually spew a warning, but only if the process
was blocked for longer than we (the kernel) expected.

> Removing it does not magically fix the underlying problem - it
> simply means that we don't hear about them until someone complains
> that unmount is taking hours....

There isn't necessarily an underlying problem.  This is very much a gray
area, Dave.  We get plenty of false positives in this code.  I was
trying to reduce *that* problem.  Do you have a better idea on how to
address the issue?

Maybe this discussion requires looking at specific instances of the
problem so we're all on the same page.  What do you think is the best
way forward, here?

Cheers,
Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/