linux-ext4 - [Bug 151491] free space lossage on busy system with bigalloc enabled and 128KB cluster

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <bug-151491-13602-6X9NJMdKhv@https.bugzilla.kernel.org/>
Date:   Thu, 30 Nov 2017 16:08:41 +0000
From:   bugzilla-daemon@...zilla.kernel.org
To:     linux-ext4@...nel.org
Subject: [Bug 151491] free space lossage on busy system with bigalloc enabled
 and 128KB cluster

https://bugzilla.kernel.org/show_bug.cgi?id=151491

--- Comment #14 from Eric Whitney (enwlinux@...il.com) ---
I've been able to identify the unknown contributor to the delalloc accounting
error as discussed previously (in comment 10).

When ext4 is writing back a previously delalloc'ed extent that contains just a
portion of a cluster, and then delallocs an extent that contains another
disjoint portion of that same cluster, the count of delalloc'ed clusters
(i_reserved_data_blocks) is incorrectly incremented.  The cluster has been
physically allocated during writeback, but the subsequent delalloc write does
not discover that allocation.  This is because the code in ext4_da_map_blocks()
checks for a previously physically allocated block at the point of allocation
rather than a previously physically allocated cluster spanning the point of
allocation.

The effect is to bump the delalloc'ed cluster count for clusters that will
never be allocated (since they've already been allocated), and the overcount
will therefore never be reduced.

It's more likely this problem would occur when writing files sequentially if
the test system was under memory pressure, resulting in writeback activity in
parallel with delalloc writes.  The magnitude of the overcount is also likely
to be larger in this situation.  This correlates well with the observation that
the reproducer for the accounting errors is more likely to reproduce the
problem on a test system with little free memory.

I've been testing a prototype patch that appears to fix this problem.  However,
I've also identified at least two other unrelated delalloc accounting problems
for bigalloc file systems whose effects are masked by the other contributor to
overcounting in ext4_ext_map_blocks().  Fixing it results in failures caused by
these other problems when running xfstests-bld on bigalloc.  So, there's a lot
of work yet to be done before it's time to post patches.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.