linux-ext4 - Re: How to force EXT4_MB_GRP_CLEAR

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <97FC9914-F57C-4D96-8C24-5B6AE7C6FE71@dilger.ca>
Date:   Mon, 25 Oct 2021 10:47:46 -0600
From:   Andreas Dilger <adilger@...ger.ca>
To:     Laurent GUERBY <laurent@...rby.net>
Cc:     Lukas Czerner <lczerner@...hat.com>, linux-ext4@...r.kernel.org
Subject: Re: How to force EXT4_MB_GRP_CLEAR_TRIMMED on a live ext4?

On Oct 25, 2021, at 09:29, Laurent GUERBY <laurent@...rby.net> wrote:
> 
> On Mon, 2021-10-25 at 11:42 +0200, Lukas Czerner wrote:
>>> On Sat, Oct 23, 2021 at 12:24:40PM +0200, Laurent GUERBY wrote:
>>> 
>>> I did end up creating dummy files to fill the filesystem and then
>>> removing them, but this is far less efficient than what a
>>> filesystem
>>> tool could do.
>> 
>> Yeah, that's bad. The information is stored in the buddy cache in
>> memory
>> and AFAIK is only dropped on unmount. I'll have to think about how to
>> clear either the cache or selectively just the flag.
>> 
>> What would be more convenient way of doing this for you, -o remount,
>> or
>> using let's say tune2fs ? I am not promising anything yet, but I'll
>> think
>> about how to implement it.
>> 
>> 
>> Meanwhile other than umount/mount, or actually writing to the dummy
>> files,
>> you can try to use fallocate to allocate all the remaining space in
>> the
>> file system and subsequently removing it. That should be more
>> efficient,
>> but don't forget to sync after remove to make sure the space is
>> released
>> before you call fstrim.
> 
> Thanks for the advice on fallocate! It does work and is very fast.

It would be enough to allocate and free a block in each group (128MB)
of the filesystem. That can't be controlled directly by fallocate(),
but indirectly via the "goal inode", but if fallocate() if all free space is fast
enough it may not be worth the effort. 

> I would prefer a specific tune2fs as remount forcing this TRIM cache
> clearing behaviour might be unwanted.
> 
>> You could also force fsck on ro file system and use -E discard to
>> trim the
>> free space but I can't say I recommend it.

This would have a danger to corrupt the mounted filesystem, so should
not be allowed when doing a read-only e2fsck. The bitmaps that e2fsck
is using for the trim may be stale if the filesystem is modified since the
start of the run, which may be a long time in some cases (minutes, hours).

Cheers, Andreas