linux-ext4 - [RFC PATCH v4 0/8] Ext4 fast commit performance patch series

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20220721060246.1696852-1-harshadshirwadkar@gmail.com>
Date:   Thu, 21 Jul 2022 06:02:38 +0000
From:   Harshad Shirwadkar <harshadshirwadkar@...il.com>
To:     linux-ext4@...r.kernel.org
Cc:     tytso@....edu, Harshad Shirwadkar <harshadshirwadkar@...il.com>
Subject: [RFC PATCH v4 0/8] Ext4 fast commit performance patch series

This is the V4 of the patch series. This patch series supersedes the
patch "ext4: remove journal barrier during fast commit" sent in Feb
2022.

The main difference from V3 is that this patch series makes use extent
status tree in the commit path. While this results in no regressions
in the "quick" group, there is a failure in "log" group (generic/311)
when fast commit is turned on. I am in the middle of getting to the
root of it, but my experiments so far suggest that there is an
inconsistency in ext4_map_blocks()'s behavior when
"EXT4_GET_BLOCKS_CACHED_NOWAIT" flag is passed even if shrinking es
tree is disabled. Due to that failure, I have converted this patch
series to RFC. I will update the series once I fix the inconsistency
issue. But, I wanted to get the current version out any way to get
feedback on the direction this patch series going and also to
potentially catch issues that I might have missed.

Original Cover Letter
---------------------

This patch series reworks the fast commit's commit path to improve
overall performance of the commit path. Following optimizations have
been added in this series:

* Avoid having to lock the journal throughout the fast commit.
* Remove tracking of open handles per inode.

With the changes introduced in this patch series, now the commit path
for fast commits is as follows:

 [1] Lock the journal by calling jbd2_journal_lock_updates. This
     ensures that all the exsiting handles finish and no new handles
     can start.
 [2] Mark all the fast commit eligible inodes as undergoing fast commit
     by setting "EXT4_STATE_FC_COMMITTING" state.
 [3] Unlock the journal by calling jbd2_journal_unlock_updates. This allows
     starting of new handles. If new handles try to start an update on
     any of the inodes that are being committed, ext4_fc_track_inode()
     will block until those inodes have finished the fast commit.
 [4] Submit data buffers of all the committing inodes.
 [5] Wait for [4] to complete.
 [6] Commit all the directory entry updates in the fast commit space.
 [7] Commit all the changed inodes in the fast commit space and clear
     "EXT4_STATE_FC_COMMITTING" for all the inodes.
 [8] Write tail tag to ensure atomicity of commits.

(The above flow has been documented in the code as well)

I verified that the patch series introduces no regressions in "quick"
and "log" groups when "fast_commit" feature is enabled.

Harshad Shirwadkar (8):
  ext4: convert i_fc_lock to spinlock
  ext4: for committing inode, make ext4_fc_track_inode wait
  ext4: use extent status tree in fast commit path
  ext4: rework fast commit commit path
  ext4: ext4_fc_track_inode() bug fix
  ext4: drop i_fc_updates from inode fc info
  ext4: add lockdep annotation in fc commit path
  ext4: update code documentation

 fs/ext4/ext4.h           |  21 ++--
 fs/ext4/extents_status.c |   3 +-
 fs/ext4/fast_commit.c    | 248 ++++++++++++++++++++-------------------
 fs/ext4/inline.c         |   3 +
 fs/ext4/inode.c          |   5 +-
 fs/ext4/super.c          |   2 +-
 fs/jbd2/journal.c        |   2 -
 7 files changed, 149 insertions(+), 135 deletions(-)

-- 
2.37.0.170.g444d1eabd0-goog