linux-ext4 - [RFC PATCH 0/5] ext4: rework delayed allocated cluster accounting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20180513175624.12887-1-enwlinux@gmail.com>
Date:   Sun, 13 May 2018 13:56:19 -0400
From:   Eric Whitney <enwlinux@...il.com>
To:     linux-ext4@...r.kernel.org
Cc:     tytso@....edu, Eric Whitney <enwlinux@...il.com>
Subject: [RFC PATCH 0/5] ext4: rework delayed allocated cluster accounting

The goals of this patch series are to solve the specific bugs
described in bugzilla #151491 and to arrive at a correct solution
for delayed allocated cluster accounting for bigalloc file systems
generally. Under some circumstances, ext4 currently makes gross
overestimates of the number of reserved clusters required to handle
bigalloc write requests under delayed allocation.  In addition to
premature ENOSPC and quota limit failures, these overestimates tend
to persist over time and in some cases even persist across umounts
and remounts of an affected file system.

This patch series is a work in progress.  Although it does appear to
correct all known delayed cluster accounting deficiencies, and
produces no regressions for all xfstest-bld test cases, it currently
contains a significant bug that adversely affects ext4's direct I/O
code.  The next and hopefully final step is to address that bug.
These patches are being posted now to show the work in progress and
to solicit comments on its general approach and implementation.

All of the patches in the series must be applied as a unit to obtain
a correct solution to the problem.  The individual fixes in this
series have not been submitted separately because they can cause
new failures or misbehavior to appear.  The effect is to make matters
even worse for the use case described in bugzilla #151491 until all
the fixes have been applied together.

This patch series is based on 4.17-rc3.  4.17-rc4 has exhibited a
number of undiagnosed xfstest-bldtest test failures that appear to be
unrelated to ext4 and which make comparisons with a regression
baseline troublesome.  4.17-rc3 is free of those failures.

Eric Whitney (5):
  ext4: fix reserved cluster accounting at delayed write time
  ext4: reduce reserved cluster count by number of allocated clusters
  ext4: adjust reserved cluster count when removing extents
  ext4:  release delayed allocated clusters when removing block ranges
  ext4: don't release delalloc clusters when invalidating page

 fs/ext4/ext4.h           |  11 +
 fs/ext4/extents.c        | 661 ++++++++++++++++++++++++++++-------------------
 fs/ext4/extents_status.c | 303 ++++++++++++++++++++++
 fs/ext4/extents_status.h |  20 ++
 fs/ext4/inode.c          | 113 ++++----
 fs/ext4/mballoc.c        |  11 +-
 6 files changed, 798 insertions(+), 321 deletions(-)

-- 
2.11.0