linux-kernel - Re: Deleting a bunch of files takes long.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250908161851.pnnbdqetb5oismhs@BitWizard.nl>
Date: Mon, 8 Sep 2025 18:18:51 +0200
From: Rogier Wolff <R.E.Wolff@...Wizard.nl>
To: Theodore Ts'o <tytso@....edu>
Cc: linux-kernel@...r.kernel.org
Subject: Re: Deleting a bunch of files takes long.

On Fri, Sep 05, 2025 at 09:11:30AM -0400, Theodore Ts'o wrote:

> There is a workaround; see the attached spd_readdir.c which you can
> use as a LD_PRELOAD.  It overrides readdir() by sorting the returned

I've now had the chance to test this. With the LD_PRELOAD I'm getting
times of "about 10 minutes" per toplevel directory to delete, and min
(quite possibly due to the "data": from 4:09 lowest to 12:05 highest and
many around the 10:10 mark). 

Without the spd_readdir, I'm seeing 5:40 as the lowest and 13:03 as
the highest. So the performance increase is "about 20%".

My technical intuition says that a factor of 2-10 should be possbile.
(so 50-90% reduction in time). (While hoping for a factor of 100, 99%
reduction in runtime).

Is the "logging file system" a contributing factor? Maybe after each
rm or after each rmdir that something needs to be written to the log?

At one point in time I accidentally deleted my whole kernel tree. That
took "longer than expected" which made me think and realize what I had
done. It was about 2 or 3 seconds. The operating system was Minix and
the PC was a PC-XT at 10MHz. I doublechecked my solution for a couple
of seconds and powered it off before the cached results (everything
deleted) could be written to disk. After the expected fsck my kernel 
sources were still there. 

I have the impression that Linux has been worse than Minix in such
cases for about three decades now.

	Roger.

-- 
** R.E.Wolff@...Wizard.nl ** https://www.BitWizard.nl/ ** +31-15-2049110 **
** Verl. Spiegelmakerstraat 37 2645 LZ  Delfgauw, The Netherlands.  
** KVK: 27239233    **
f equals m times a. When your f is steady, and your m is going down
your a** is going up.  -- Chris Hadfield about flying up the space shuttle.
**  'a' for accelleration.