linux-kernel - Re: ext3/ext4 directories don't shrink after deleting lots of files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 16 May 2009 02:42:17 -0700 (PDT)
From:	david@...g.hm
To:	Theodore Tso <tytso@....edu>
cc:	Timo Sirainen <tss@....fi>, Josef Bacik <josef@...icpanda.com>,
	linux-kernel@...r.kernel.org
Subject: Re: ext3/ext4 directories don't shrink after deleting lots of
 files

On Fri, 15 May 2009, Theodore Tso wrote:

> On Thu, May 14, 2009 at 08:45:38PM -0400, Timo Sirainen wrote:
>>
>> I was rather thinking something that I could run while the system was
>> fully operational. Otherwise just moving the files to a temp directory +
>> rmdir() + rename() would have been fine too.
>>
>> I just tested that xfs, jfs and reiserfs all shrink the directories
>> immediately. Is it more difficult to implement for ext* or has no one
>> else found this to be a problem?
>
> It's probably fairest to say no one has thought it worth the effort.
> It would require some fancy games to swap out block locations in the
> extent trees (life would be easier with non-extent-using inodes), and
> in the case of htree, we would have to keep track of the index block
> so we could remove it from the htree index.  So it's all doable, if a
> bit tricky in terms of the technical details; it's just that the
> people who could do it have been busy enough with other things.
>
> It's hasn't been considered high priority because most of the time
> directories don't go from holding thousands of files down to a small
> handful.

I see it on a fairly regular basis on mail servers. in sendng a large 
queue builds up due to a remote system being down, but then after the 
remote system recovers, all access to that directory is slow, hurting 
everything else on the system.

David Lang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/