linux-ext4 - Re: Make huge files strictly contiguous (fallocate, bigalloc, e4defrag...)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 20 Apr 2014 15:01:46 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Contact <neitsab@....fr>
Cc:	Lukáš Czerner <lczerner@...hat.com>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: Make huge files strictly contiguous (fallocate, bigalloc,
 e4defrag...)

On Sun, Apr 20, 2014 at 04:37:54PM +0200, Contact wrote:
> While we're at it, how can I do this? I searched the 
> mkfs.ext4/mke2fs.conf/ext4/tune2fs man pages for the option to turn off 
> writing backup superblock you mentioned, however the closest I found is 
> tune2fs -O ^sparse_super which is part of the defaults mke2fs.conf 
> options... Could you point me at it so that I can try to get this 
> source of non-contiguity away?

This requires recompiling e2fsprogs using the tip of the maint branch,
as I had suggested earlier (and which you had ruled out because you
didn't want to build your own e2fsprogs from source).

Once you do this, and you put the following in 

	hugefiles = {
		features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
		hash_alg = half_md4
		num_backup_sb = 0
		packed_meta_blocks = 1
		flex_bg_size = 262144
		inode_ratio = 4194304
		make_hugefiles = 1
		hugefiles_dir = /boot
		hugefiles_name = contig-files
		hugefiles_digits = 1
		hugefiles_size = 256M
		num_hugefiles = 3
	}

Using the sparse_super2 feature and setting num_backup_sb=0 disables
the backup superblocks.  Then setting flex_bg_size to a very large
value (262144 in this case), will force all of the allocation bitmaps
and inodetables to the very beginning of the file system.

The inode_ratio setting simply radically reduces the number of inodes
in the file system.  That was there because originally I was using
this this command set with the hugefiles_size set to 1 or 2 gigabytes,
and filling the disk with completely with large files (num_hugefiles =
0 means "create as many huge files as necessary to fill the disk). 

The rest of the "hugefiles" related settings are just simply allowing
mke2fs to create the contiguous files for you.

Note that because this uses a new compat feature, "sparse_super2", if
you use the new mke2fs to create such a file system, you will also
need to make sure the file system is checked using a version of e2fsck
that supports this new feature as well.  No kernel changes are
required to support this, though.

Cheers,

					- Ted

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html