linux-ext4 - Re: About reserve of blocks for "overflow extents" in ext4 metadata

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4B1E754E.6080505@redhat.com>
Date:	Tue, 08 Dec 2009 09:48:30 -0600
From:	Eric Sandeen <sandeen@...hat.com>
To:	Vyacheslav Dubeyko <Vyacheslav.Dubeyko@...onis.com>
CC:	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: About reserve of blocks for "overflow extents" in ext4 metadata

Vyacheslav Dubeyko wrote:
> Hello,
> 
> I think that it make sense to has in ext4 metadata a reserve of
> blocks for "overflow extents" (it is the extents that to form
> extent's tree and it is placed in some blocks is described in i_block
> inode's field for a file). The reserve of blocks for "overflow
> extents" can be located (during operation of ext4 file system
> creation by mkfs) after inode table for every virtual (FLEX_BG) group
> by united aggregate of blocks. The size and placement of this reserve
> has to be described by free special inode.
> 
> In my opinion, the reserve of blocks for "overflow extents" resolves
> such problems: 1) In the case of ext4 volume's shrinking resize
> (especially, in the case of very fragmented volume) it can be very
> difficult to estimate possibility of successful resize because of
> existing mechanism of extents' tree layout on the volume. It is
> possible to encounter during resize the problem of free blocks' lack
> for rebuilding of extents' tree for replaced files. The reserve of
> blocks for "overflow extents" guarantee against encountering of such
> problem during resizes. 2) The presence of the reserve of blocks for
> "overflow extents" means that all existing extents' trees of files
> will locate in one place. This fact and placement the reserve just
> after inode table will increase efficiency of operations with
> extents' trees, in my opinion. 3) The localized layout of extents'
> trees of files means efficient journaling of this metadata, also.
> 
> I think that the reserve of blocks for "overflow extents" can has
> such on-disk layout. The reserve is union of bitmap (that keeps
> knowledge about used and free blocks in reserve) and some number of
> blocks (used for extents' trees). All blocks has allocated for the
> reserve during volume creation has to set as used in block bitmap of
> group(s) that contains the reserve. The size in blocks of the reserve
> can be defined by: inode_counts * count_blocks_for_inode (count of
> blocks that make possible to form extents' tree with some average
> depth). The field i_block of special inode (that will describe the
> reserve) will have two extents: 1) the extent that describes
> placement and size of reserve's bitmap block(s); 2) the extent that
> describes placement and size of blocks used for trees' extents.

If I understand this correctly, then you would be pre-reserving
all extent metadata blocks that are possible on the filesystem, in 
the same way that we currently pre-provision inodes, at mkfs time?

What happens if we have a highly fragmented filesystem, and we 
run out of these reserved "overflow extents" blocks?  And would
overprovisioning waste more filesystem space as the inodes do
today?

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html