linux-ext4 - [PATCH 0/2] make jbd2 support checkpoint asynchronously

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 22 Sep 2019 15:04:57 +0800
From:   Xiaoguang Wang <xiaoguang.wang@...ux.alibaba.com>
To:     linux-ext4@...r.kernel.org
Subject: [PATCH 0/2] make jbd2 support checkpoint asynchronously

In current jbd2's implemention, jbd2 won't reclaim journal space unless
free journal space is lower than specified threshold, see logic in
add_transaction_credits():
        write_lock(&journal->j_state_lock);
        if (jbd2_log_space_left(journal) < jbd2_space_needed(journal))
            __jbd2_log_wait_for_space(journal);
        write_unlock(&journal->j_state_lock);

Indeed with this logic, we can also have many transactions queued to be
checkpointd, which means these transactions still occupy jbd2 space.

Journal space is somewhat like a global lock. In high concurrency case,
if many tasks contend for journal credits, they will easily be blocked in
waitting for free journal space, so I wonder whether we can reclaim journal
space asynchronously when free space is lower than a specified threshold,
to avoid that all applications are stalled at the same time. This will be
more useful in high speed store, journal space will be reclaimed in background
quickly, and applications will less likely to be stucked,  to improve this
case, we use workqueue to queue a work in background to reclaim journal space.

I have used fs_mark to have performance test, in most cases, we have performance
improvement, in specific case, we can have above 14.4% improvement, see patch
"ext4: add async_checkpoint mount option" for detailed test info.

Xiaoguang Wang (2):
  jbd2: checkpoint asynchronously when free journal space is lower than
    threshold
  ext4: add async_checkpoint mount option

 fs/ext4/ext4.h        |  2 ++
 fs/ext4/super.c       | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/jbd2/checkpoint.c  | 28 +++++++++++++++++---
 fs/jbd2/journal.c     | 15 +++++++++--
 fs/jbd2/transaction.c | 16 ++++++++++++
 include/linux/jbd2.h  | 48 +++++++++++++++++++++++++++++++++-
 6 files changed, 174 insertions(+), 6 deletions(-)

-- 
1.8.3.1