linux-ext4 - Re: Latency writing to an mlocked ext4 mapping

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Wed, 2 Nov 2011 21:17:07 +0100
From:	Jan Kara <jack@...e.cz>
To:	Andy Lutomirski <luto@...capital.net>
Cc:	Jan Kara <jack@...e.cz>, Andreas Dilger <adilger@...ger.ca>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: Latency writing to an mlocked ext4 mapping

On Tue 01-11-11 18:51:04, Andy Lutomirski wrote:
> On Tue, Nov 1, 2011 at 4:10 PM, Andy Lutomirski <luto@...capital.net> wrote:
> > On Tue, Nov 1, 2011 at 4:03 PM, Jan Kara <jack@...e.cz> wrote:
> >> Avoiding IO during a minor fault would be a decent thing which might be
> >> worth pursuing. As you properly noted "stable pages during writeback"
> >> requirement is one obstacle which won't be that trivial to avoid though...
> >
> > There's an easy solution that would be good enough for me: add a mount
> > option to turn off stable pages.
> >
> > Is the other problem just a race, perhaps?  __block_page_mkwrite calls
> > __block_write_begin (which calls get_block, which I think is where the
> > latency comes from) *before* wait_on_page_writeback, which means that
> > there might not be any space allocated yet.
> 
> I think I'm right (other than calling it a race).  If I change my code to do:
> 
> - map the file (with MCL_FUTURE set)
> - fallocate
> - dirty all pages
> - fsync
> - dirty all pages again
> 
> in the non-real-time thread, then a short test that was a mediocre
> reproducer seems to work.
> 
> This is annoying, though -- I'm not generating twice as much write I/O
> as I used to.  Is there any way to force the delalloc code to do its
> thing without triggering writeback?  I don't think fallocate has this
> effect.
  fallocate() will preallocate blocks on disk backing the mapped page. That
should get rid of latency in __block_write_begin(). Extents will still be
marked as uninitialized, but conversion from uninitialized to initialized
state happens during writeback / IO completion so you should not care much
about it.

									Honza
-- 
Jan Kara <jack@...e.cz>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html