linux-ext4 - Re: Where all does preallocated/extra space hide?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4ACE0BA9.4090607@redhat.com>
Date:	Thu, 08 Oct 2009 10:56:25 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	"Aneesh Kumar K.V" <aneesh.kumar@...ux.vnet.ibm.com>
CC:	ext4 development <linux-ext4@...r.kernel.org>
Subject: Re: Where all does preallocated/extra space hide?

Aneesh Kumar K.V wrote:
> On Tue, Sep 29, 2009 at 11:47:19AM -0500, Eric Sandeen wrote:
>> I was running some of the xfstests enospc tests on ext4, and they were
>> failing; in one case, manymanymany small files are made to fill up a
>> 100M filesystem.  ext4 stops quite early with -ENOSPC, but after a bit,
>> (or after a "sync") we get 40MB free again.  So 40% of the fs space is
>> hidden somewhere in preallocation...
>>
>> I tried calling out to discard group prealloc but that's only a few
>> blocks.  I'll go trace through the sync paths to see what all gets
>> released, but if anyone knows offhand where the rest of that space is
>> hiding, please give me a shout.  :)
>>
> 
> 
> preallocation space is discarded by default if we fail a block allocation
> ext4_mb_discard_preallocations does that. What might be happening is the
> extra meta data blocks that we reserve for making sure we will be able
> to properly insert the new extent on block allocation.  I guess we should
> force a data allocation when we fail with ENOSPC in ext4_da_writepages
> We currently force a journal commit so that the we claim back the blocks
> from deleted files. But we can also force block allocation for delayed
> allocated inodes so that we free some of the extra meta data we reserved
> 
> -aneesh

Yep, I should have followed up, I narrowed it down to just that - the
worst-case metadata blocks - 2 metadata blocks for a 20-byte write into
an empty file.  :)

I'm working on an inode walker to push out delalloc files on enospc.

Thanks,

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html