Message-Id: <12FEF208-5FAE-4EE7-93D1-34359A0CBE4F@dilger.ca>
Date:   Mon, 11 Feb 2019 18:25:00 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     Vijay Chidambaram <vijayc@...xas.edu>
Cc:     linux-ext4@...r.kernel.org, jesus.palos@...xas.edu,
        Theodore Tso <tytso@....edu>
Subject: Re: Selective Data Journaling in ext4

On Feb 11, 2019, at 5:14 PM, Vijay Chidambaram <vijayc@...xas.edu> wrote:
> 
> Hi all,
> 
> We would like to present an idea to improve the performance of data
> journaling in ext4. Data journaling is expensive because data is
> written twice: once to the journal and once to the actual file system.
> Passing data through the journal provides consistency guarantees that
> ordered journaling mode cannot provide (for example, data journaling
> prevents a data block from being partially written).
> 
> The idea behind Selective Data Journaling is simple: create a new
> journaling mode by modifying ordered journaling mode to journal data
> blocks which are already part of a file. Data blocks which are newly
> allocated are not part of the journal, and are written out before the
> journal blocks in accordance with ordered mode's ordering guarantees.
> If there is a crash before transaction commit, the only side effect is
> un-allocated data blocks getting written with new data.
> 
> Selective Data Journaling provides a lot of the benefits of data
> journaling, at significantly lower cost. For workloads which mostly
> deal with new data blocks (any applications which update files via
> atomic rename), Selective Data Journaling can increase performance
> significantly.

One major caveat here is that files are *very rarely* overwritten in
place.  This is mostly useful for database-type workloads, and most
databases already have their own transaction journal independent of
the filesystem journal, so AFAIK this would not be a very widely-used
feature.

That said, a related, but IMHO much more useful form of selective data
journaling would be for "random IOPS" workloads, where there may be
many small writes either to a single file or to many small files, and
this IO could be aggregated and optimized with fast linear writes to
the journal, possibly on a separate flash device.  That avoids a lot
of seeks for the main filesystem device for small IO (which would
otherwise be IOPS limited and not bandwidth limited, so the double data
writes are not a limiting factor), while allowing large writes to go
directly to the filesystem device and avoid the double writes (which
would otherwise reduce IO bandwidth by half).

Since we already have delalloc to pre-stage the dirty pages before the
write, we can make a good decision about whether the file data should
be written to the journal or directly to the filesystem.

This could likely leverage the work that was already done for SMR journal
mode (Ted has patches, and I think they are available online as well),
and hopefully integrate both those patches and this new work into mainline
ext4.

I'm happy to discuss this further if you are interested.

> We came up with Selective Data Journaling during my PhD at the
> University of Wisconsin Madison [1]. I haven't inspected the ext4
> codebase deeply since then, so this optimization may already exist.
> There may also be problems with this approach that we have not
> considered -- we are open to discussion. It may also be the case that
> nobody uses data journaling so the extra complexity is not worth it.
> 
> If this is something you would like to see implemented, my student
> Jesus Palos (cced) is interested in doing so. We would like to discuss
> how best to implement this if you are interested.
> 
> [1] http://research.cs.wisc.edu/adsl/Publications/optfs-sosp13.pdf
> 
> Thanks,
> Vijay Chidambaram, UT Austin
> http://www.cs.utexas.edu/~vijay/


Cheers, Andreas






Download attachment "signature.asc" of type "application/pgp-signature" (874 bytes)

Powered by blists - more mailing lists