linux-ext4 - Re: [RFC] call end_page_writeback after converting unwritten extents in ext4_end

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Fri, 11 Jan 2013 10:29:07 +0800
From:	Zheng Liu <gnehzuil.liu@...il.com>
To:	Jan Kara <jack@...e.cz>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: [RFC] call end_page_writeback after converting unwritten extents
 in ext4_end_io

On Thu, Jan 10, 2013 at 03:47:19PM +0100, Jan Kara wrote:
[cut...]
> > Now I have an idea to solve this problem.  We start a journal before submitting
> > an io request rather than start it in ext4_convert_unwritten_extents().  The
> > reason of starting a journal in ext4_convert_unwritten_extents() is that we need
> > to calculate credits for journal.  But as far as I understand the credits is not
> > increased in this function because we have splitted extents before submitting
> > this io request.  A 'handle_t *handle' will be added into ext4_io_end_t, and it
> > will be used in ext4_convert_unwritten_extents().  Then we can avoid to
> > trigger a journal commit when starting a journal.
>   I'm actually already working on a solution. The disadvantage of starting
> a transaction before IO submission is that the handle will hold transaction
> open all the time until IO is finished and extent converted. So it can
> effectively block any filesystem activity for a relatively long time. I've
> already written a patch for JBD2 to allow transaction reservation - it
> reserves blocks in the journal but they are not attached to a particular
> transaction. Later during extent conversion we transform this reservation
> into a real handle (without waiting for the journal so locking is OK).

Great!

> 
> The part I'm missing so far is adding transaction reservation into IO
> submission path. That is actually somewhat tricky because we have to do it
> before taking page locks and propagate the reserved handle all the way down
> to the point where we allocate io_end. And furthermore we have to somehow
> deal with the fact that IO to one extent can be split among multiple BIOs
> (as it happens e.g. when an extent is longer than 512 KB which is usual
> limit on BIO size) and thus multiple io_end structures are created and
> extent is converted in parts (actually we didn't think about this problem
> previously in extent conversion code).  We don't know in advance how much
> BIOs we'll need to write the extents (bio_add_page() decides when the BIO
> is full and there are other constraints on BIO than just the total size) so
> what we need to do it so have one io_end structure shared by all the BIOs
> covering the extent. That will also save us from unnecessary splitting and
> joining of extents for conversion. But doing that requires some changes to
> io submission path which is why it's taking me longer than I'd like (plus I
> have other obligations than just improving ext4 ;) But I'm working on it so
> please stay tuned...

Now all I need to do is to have a cup of coffee and wait your patches. :)

Regards,
                                                - Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html