lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Sat, 17 Apr 2010 20:32:39 -0400
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Mel Gorman <mel@....ul.ie>
Cc:	Dave Chinner <david@...morbit.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Chris Mason <chris.mason@...cle.com>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH] mm: disallow direct reclaim page writeback


There are two issues here: stack utilisation and poor IO patterns in
direct reclaim.  They are different.

The poor IO patterns thing is a regression.  Some time several years
ago (around 2.6.16, perhaps), page reclaim started to do a LOT more
dirty-page writeback than it used to.  AFAIK nobody attempted to work
out why, nor attempted to try to fix it.


Doing writearound in pageout() might help.  The kernel was in fact was
doing that around 2.5.10, but I took it out again because it wasn't
obviously beneficial.

Writearound is hard to do, because direct-reclaim doesn't have an easy
way of pinning the address_space: it can disappear and get freed under
your feet.  I was able to make this happen under intense MM loads.  The
current page-at-a-time pageout code pins the address_space by taking a
lock on one of its pages.  Once that lock is released, we cannot touch
*mapping.

And lo, the pageout() code is presently buggy:

		res = mapping->a_ops->writepage(page, &wbc);
		if (res < 0)
			handle_write_error(mapping, page, res);

The ->writepage can/will unlock the page, and we're passing a hand
grenade into handle_write_error().

Any attempt to implement writearound in pageout will need to find a way
to safely pin that address_space.  One way is to take a temporary ref
on mapping->host, but IIRC that introduced nasties with inode_lock. 
Certainly it'll put more load on that worrisomely-singleton lock.


Regarding simply not doing any writeout in direct reclaim (Dave's
initial proposal): the problem is that pageout() will clean a page in
the target zone.  Normal writeout won't do that, so we could get into a
situation where vast amounts of writeout is happening, but none of it
is cleaning pages in the zone which we're trying to allocate from. 
It's quite possibly livelockable, too.

Doing writearound (if we can get it going) will solve that adequately
(assuming that the target page gets reliably written), but it won't
help the stack usage problem.


To solve the IO-pattern thing I really do think we should first work
out ytf we started doing much more IO off the LRU.  What caused it?  Is
it really unavoidable?


To solve the stack-usage thing: dunno, really.  One could envisage code
which skips pageout() if we're using more than X amount of stack, but
that sucks.  Another possibility might be to hand the target page over
to another thread (I suppose kswapd will do) and then synchronise with
that thread - get_page()+wait_on_page_locked() is one way.  The helper
thread could of course do writearound.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists