linux-kernel - Re: per-bdi-throttling: synchronous writepage doesn't work correctly

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1193942132.27652.331.camel@twins>
Date:	Thu, 01 Nov 2007 19:35:32 +0100
From:	Peter Zijlstra <peterz@...radead.org>
To:	Miklos Szeredi <miklos@...redi.hu>
Cc:	jdike@...toit.com, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org,
	Christoph Hellwig <hch@...radead.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Al Viro <viro@...iv.linux.org.uk>
Subject: Re: per-bdi-throttling: synchronous writepage doesn't work
	correctly

On Thu, 2007-11-01 at 18:16 +0100, Peter Zijlstra wrote:
> On Thu, 2007-11-01 at 18:09 +0100, Peter Zijlstra wrote:
> > On Thu, 2007-11-01 at 18:00 +0100, Miklos Szeredi wrote:
> > > > > Hi,
> > > > > 
> > > > > It looks like bdi_thresh will always be zero if filesystem does
> > > > > synchronous writepage, resulting in very poor write performance.
> > > > > 
> > > > > Hostfs (UML) is one such example, but there might be others.
> > > > > 
> > > > > The only solution I can think of is to add a set_page_writeback();
> > > > > end_page_writeback() pair (or some reduced variant, that only does
> > > > > the proportions magic).  But that means auditing quite a few
> > > > > filesystems...
> > > > 
> > > > Ouch...
> > > > 
> > > > I take it there is no other function that is shared between all these
> > > > writeout paths which we could stick a bdi_writeout_inc(bdi) in?
> > > 
> > > No, and you can't detect it from the callers either I think.
> > 
> > The page not having PG_writeback set on return is a hint, but not fool
> > proof, it could be the device is just blazing fast.
> > 
> > I guess there is nothing to it but for me to grep writepage and manually
> > look at all hits...
> 
>   writepage: called by the VM to write a dirty page to backing store.
>       This may happen for data integrity reasons (i.e. 'sync'), or
>       to free up memory (flush).  The difference can be seen in
>       wbc->sync_mode.
>       The PG_Dirty flag has been cleared and PageLocked is true.
>       writepage should start writeout, should set PG_Writeback,
>       and should make sure the page is unlocked, either synchronously
>       or asynchronously when the write operation completes.
> 
>       If wbc->sync_mode is WB_SYNC_NONE, ->writepage doesn't have to
>       try too hard if there are problems, and may choose to write out
>       other pages from the mapping if that is easier (e.g. due to
>       internal dependencies).  If it chooses not to start writeout, it
>       should return AOP_WRITEPAGE_ACTIVATE so that the VM will not keep
>       calling ->writepage on that page.
> 
>       See the file "Locking" for more details.
> 
> 
> The "should set PG_Writeback" bit threw me off I guess.

Hmm, set_page_writeback() is also the one clearing the radix tree dirty
tag. So if that is not called, we get in a bit of a mess, no?

Which makes me think hostfs is buggy.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/