linux-ext4 - Re: [PATCH 0/5 v2] add extent status tree caching

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130719025934.GE17938@thunk.org>
Date:	Thu, 18 Jul 2013 22:59:34 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Eric Sandeen <sandeen@...hat.com>
Cc:	Ext4 Developers List <linux-ext4@...r.kernel.org>,
	Zheng Liu <gnehzuil.liu@...il.com>
Subject: Re: [PATCH 0/5 v2] add extent status tree caching

On Thu, Jul 18, 2013 at 07:56:45PM -0500, Eric Sandeen wrote:
> > The problem is we don't know that we're doing AIO until we see the
> > first io_submit(2) call.  With this patch series, we'll pull the
> > contents of the entire leaf tree block into extent cache, but if the
> > extent tree is larger than that, if we read in the entire extent tree
> > on the first AIO request, then that first request will delayed even
> > more, and it's not clear that's a good thing.
> 
> Is blocking on a pre-AIO ioctl better than blocking on the
> first AIO?

The precache ioctl is something which the application is expecting to
block.  The question is, if we have a heavily fragmented extent tree,
is it better for the first AIO to block long enough to read in one
metadata block --- and then never block again, or to have that first
AIO request take a long, LONG time?  Especially if the application
isn't expecting it?

Also there are use cases for the precache ioctl even if you are not
using AIO.  If you've taken care to make sure the file is as
contiguous as possible, having the extents be cached will save a lot
of memory compared to if the buffer heads are always entering the
buffer cache.  So reading in all of the metadata can be a good thing
to do, especially if you can do this *before* you declare that the
server is healthy and is ready to start receiving traffic.

This becomes especially critical if the server is running in a very
tight memory container, because you are trying to pack as many jobs
(or VM's, if that's the way you roll) as possible on a physical
server.

Cheers,

						- Ted

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html