linux-ext4 - Re: [RFC] Add new extent structure in ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4F270091.3050000@redhat.com>
Date:	Mon, 30 Jan 2012 14:41:53 -0600
From:	Eric Sandeen <sandeen@...hat.com>
To:	Robin Dong <hao.bigrat@...il.com>
CC:	"Ted Ts'o" <tytso@....edu>, Andreas Dilger <adilger@...ger.ca>,
	Ext4 Developers List <linux-ext4@...r.kernel.org>
Subject: Re: [RFC] Add new extent structure in ext4

On 1/23/12 6:51 AM, Robin Dong wrote:
> Hi Ted, Andreas and the list,
> 
> After the bigalloc-feature is completed in ext4, we could have much more
> big size of block-group (also bigger continuous space), but the extent
> structure of files now limit the extent size below 128MB, which is not
> optimal.
> 
> We could solve the problem by creating a new extent format to support
> larger extent size, which looks like this:
> 
> struct ext4_extent2 {
> 	__le64	ee_block;	/* first logical block extent covers */
> 	__le64	ee_start;	        /* starting physical block */
> 	__le32	ee_len;		/* number of blocks covered by extent */
> 	__le32	ee_flags;	/* flags and future extension */
> };
> 
> struct ext4_extent2_idx {
> 	__le64	ei_block;	        /* index covers logical blocks from 'block' */
> 	__le64	ei_leaf;	        /* pointer to the physical block of the next level */
> 	__le32	ei_flags;	        /* flags and future extension */
> 	__le32	ei_unused;	/* padding */
> };
> 
> I think we could keep the structure of ext4_extent_header and add new
> imcompat flag EXT4_FEATURE_INCOMPAT_EXTENTS2.
> 
> The new extent format could support 16TB continuous space and larger volumes.

(larger volumes?)

> What's your opinion?
> 

I think that mailing list drama aside ;) Dave has a decent point that we shouldn't
allow structures to scale out further than the code *using* them can scale.

In other words, if we already have some trouble being efficient with 2^32 blocks
in a file, it is risky and perhaps unwise to allow even larger files, until
those problems are resolved.  At a minimum, I'd suggest that such a change
should not go in until it is demonstrated that ext4 can, in general, handle such
large file sizes efficiently.

It'd be nice to be able to self-host large sparse images for large fs testing,
though.

I suppose bigalloc solves that a little, though with some backing store space
usage penalty.  I suppose if a bigalloc fs is hosted on a bigalloc fs, things
should (?) line up and be reasonable.

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html