Date:   Fri,  1 May 2020 11:59:42 +0530
From:   Ritesh Harjani <riteshh@...ux.ibm.com>
To:     linux-ext4@...r.kernel.org
Cc:     "Paul E . McKenney" <paulmck@...nel.org>,
        linux-fsdevel@...r.kernel.org, Jan Kara <jack@...e.com>,
        tytso@....edu, "Aneesh Kumar K . V" <aneesh.kumar@...ux.ibm.com>,
        Ritesh Harjani <riteshh@...ux.ibm.com>
Subject: [RFC 00/20] ext4: Fix ENOSPC error, improve mballoc dbg, other cleanups

Hello All,

v2 -> v3:
v3 changes the code design to fix ENOSPC error. This patch uses the percpu
discard pa seq counter (which was as discussed with Jan & Aneesh).
Patch-2 commit msg describes both the problem and the new algorithm in
great detail. Rest of the patches are mostly either refactoring, code
cleanups or debug logs improvements.

Posting this for early review comments.
For now I have done some basic testing on this patch to test
for ENOSPC reported errors, using a smaller filesystem (~240MB, 64K bs)
a. Tested multi-thread file writes which only allocates group PAs.
b. Tested multi-thread file writes which only allocates inode PAs.
c. Tested multi-thread file writes doing combination of both of the above.

[May Not be Ready for Merge yet, until below are properly discussed]
==================================================================

1. There is a query asked in Patch-2 commit msg itself, regarding
   rcu_barrier() usage.

2. AFAICT, even if we reduce the PA size based on available FS size to avoid
   this ENOSPC error, this issue could still potentially happen since the race
   has mostly to do with 1st thread freeing up all the PAs while 2nd thread
   returning 0 as freed (since PA list was empty). Hence 2nd thread fails
   with ENOSPC even though there were free blocks freed by 1st thread.

3. I would like to know if there is any stress-ng test case or any other use
   case which measures multi-thread performance of write while FS is close to
   ENOSPC? If yes - instead of regressing this later, I would like to know
   such test case so that it can be tested at my end while we are still
   at it.

4. I do see that with 64K blocksize the performance of multi-thread < 1 MB
   file size writes becomes very slow. But without this patch, it anyways fails
   with ENOSPC error. On doing below the performance does improve close to 5x.
   echo 32 > /sys/fs/ext4/loop3/mb_group_prealloc
   I was thinking instead of 512 blocks as default value for
   'MB_DEFAULT_GROUP_PREALLOC', we could make it 64K as the default size and
   decide the no. of blocks based on the blocksize. But I haven't got to it
   yet. But thought to capture it in this email though. We can get to it
   after these patches.

5. Started fstests testing. Will let you know the results soon.

[RFCv2]: https://patchwork.ozlabs.org/project/linux-ext4/patch/533ac1f5b19c520b08f8c99aec5baf8729185714.1586954511.git.riteshh@linux.ibm.com/
         - Problems with v2 are captured in Patch-2 commit msg itself.


Ritesh Harjani (20):
  ext4: mballoc: Refactor ext4_mb_discard_preallocations()
  ext4: Introduce percpu seq counter for freeing blocks(PA) to avoid
    ENOSPC err
  ext4: mballoc: Do print bb_free info even when it is 0
  ext4: mballoc: Refactor ext4_mb_show_ac()
  ext4: mballoc: Add more mb_debug() msgs
  ext4: mballoc: Correct the mb_debug() format specifier for pa_len var
  ext4: mballoc: Fix few other format specifier in mb_debug()
  ext4: mballoc: Simplify error handling in ext4_init_mballoc()
  ext4: mballoc: Make ext4_mb_use_preallocated() return type as bool
  ext4: mballoc: Remove EXT4_MB_HINT_GOAL_ONLY and it's related code
  ext4: mballoc: Refactor code inside DOUBLE_CHECK into separate
    function
  ext4: mballoc: Fix possible NULL ptr dereference from mb_cmp_bitmaps()
  ext4: mballoc: Don't BUG if kmalloc or read blk bitmap fail for
    DOUBLE_CHECK
  ext4: balloc: Use task_pid_nr() helper
  ext4: Use BIT() macro for BH_** state bits
  ext4: Improve ext_debug() msg in case of block allocation failure
  ext4: Replace EXT_DEBUG with __maybe_unused in
    ext4_ext_handle_unwritten_extents()
  ext4: mballoc: Make mb_debug() implementation to use pr_debug()
  ext4: Make ext_debug() implementation to use pr_debug()
  ext4: Add process name and pid in ext4_msg()

 fs/ext4/Kconfig             |   3 +-
 fs/ext4/balloc.c            |   5 +-
 fs/ext4/ext4.h              |  38 ++--
 fs/ext4/extents.c           | 150 ++++++++--------
 fs/ext4/inode.c             |  15 +-
 fs/ext4/mballoc.c           | 347 +++++++++++++++++++++++-------------
 fs/ext4/mballoc.h           |  16 +-
 fs/ext4/super.c             |   3 +-
 include/trace/events/ext4.h |   1 -
 9 files changed, 335 insertions(+), 243 deletions(-)

-- 
2.21.0

Powered by blists - more mailing lists