linux-kernel - Re: Approaches to making io

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAAK6Zt2P3jBPfeCPioK-ehpVAw0iQjikY0BujXfQmfozJ8GNvA@mail.gmail.com>
Date:	Tue, 30 Aug 2011 14:51:01 -0700
From:	Daniel Ehrenberg <dehrenberg@...gle.com>
To:	Christoph Hellwig <hch@...radead.org>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: Approaches to making io_submit not block

Thanks for getting back to me.

On Mon, Aug 29, 2011 at 10:32 PM, Christoph Hellwig <hch@...radead.org> wrote:
> On Mon, Aug 29, 2011 at 10:33:24AM -0700, Daniel Ehrenberg wrote:
>> - Blocking due to reading metadata.
>> Proposed solution:
>> Add a per-ioctx work queue to do metadata reads. It will be triggered
>> from the dio code: if in async mode, then get_block will be called
>> with an additional flag, meaning something like O_NONBLOCK on sockets.
>> File systems' get_block functions can implement this flag and return
>> -EAGAIN if a read from the underlying device would be necessary. (If
>> we're worried that EAGAIN might be used for other purposes in the
>> future, we could make a new errno for this purpose.) From a quick
>> glance at the code, it looks like this would not be too difficult to
>> add to ext4 for extent-based files, and support in other file systems
>> could be added gradually. If -EAGAIN is returned, then the struct dio
>> will be put on the work queue together with a description of what kind
>> of processing it was doing. The work queue only serves the metadata
>> request, and the rest of the request is served on the existing path.
>
> Let filesystems handle this.  I've actually prototyped it in XFS,
> based on some pending work from Dave but at this point it's still butt
> ugly.

Great, would you be willing to let me see the draft code?

Are you sure that there wouldn't be any benefit to having the code be
in the aio/dio levels in terms of making it easier for file
systems/reducing code duplication?
>
>> - Blocking for appends and writes to file holes due to the need for a
>> metadata write after the data write
>> Proposed solution:
>> Maintain a work queue for all appends and writes to file holes, which
>> executes the current code.
>
> No way.  I've fixed this for XFS, and it's trivial without the need to
> queue them up.  The only thing preventing appending writes to work is
> a flag to tell the dio layer to just do them, just like it already works
> for holes.  (and more QA).
>
>
Are you saying this is already fixed for XFS? Appends don't block,
only reads to metadata do?

Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/