linux-kernel - Re: khugepaged: gets stuck when writing to USB flash, 2.6.38-rc2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110203132407.GF11958@csn.ul.ie>
Date:	Thu, 3 Feb 2011 13:24:08 +0000
From:	Mel Gorman <mel@....ul.ie>
To:	Andrea Arcangeli <aarcange@...hat.com>
Cc:	Jind??ich Makovi??ka <makovick@...il.com>,
	linux-kernel@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: khugepaged: gets stuck when writing to USB flash, 2.6.38-rc2

On Tue, Feb 01, 2011 at 04:49:47PM +0100, Andrea Arcangeli wrote:
> On Mon, Jan 31, 2011 at 08:28:00PM +0100, Jind??ich Makovi??ka wrote:
> > Hi,
> > 
> > I am encountering problems when continuously writing larger amounts of
> > data to a USB flash drive. My configuration is
> > 
> > x86-64 kernel
> > USB stick with 10MB/s write, 30MB/s read speed,
> > HDD with ~60-80MB/s read/write
> > 8 GiB RAM
> > 
> > When copying 4GB or more in one go from HDD to Flash, during the
> > copying, fork() and probably other syscalls involving VM start
> > blocking (I first observed the problem in Chrome, which refused to
> > display content in new tabs). When one lets the copying finish, the
> > system returns to a usable state.
> > 
> > During the limbo, khugepaged is in D state (uninterruptible sleep).
> 
> That means no hugepage could be allocated. Maybe memory compaction is
> doing an overwork because all pagecache is dirty and can't be
> migrated. This should solve it if it's memory compaction:
> 

This is very likely. Compaction calls into migration which will wait on
dirty pages after a time. With a large number of dirty pages backed by a
slow drive such as a USB stick, it could be getting stalled there for a
long period of time. 

Whether migration sleeps or not can be controlled by the sync parameter
passed into try_to_compact_memory which could be always forced to false
if GFP_NO_KSWAPD?

> echo never >/sys/kernel/mm/transparent_hugepage/defrag
> 
> kswapd state would be interesting too. Can you sysrq+t?
> 
> Probably we should decrease the aggressiveness of memory compaction in
> direct reclaim. I've another report that memory compaction for order <
> 3 allocations is increasing latency, it's not like your problem but it
> may be related. The congestion_wait in compaction.c also makes me
> uncomfortable, it should bail out and fail I think. Maybe we should
> add a bitflag to differentiate the callers that can gracefully handle
> failure (like THP or most skb jumbo frame allocations) and those like
> the kernel stack that will return -ENOMEM if allocation fails.
> 

-- 
Mel Gorman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/