linux-ext4 - Re: Make huge files strictly contiguous (fallocate, bigalloc, e4defrag...)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140417152457.GE18591@thunk.org>
Date:	Thu, 17 Apr 2014 11:24:57 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Contact <neitsab@....fr>
Cc:	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: Make huge files strictly contiguous (fallocate, bigalloc,
 e4defrag...)

On Thu, Apr 17, 2014 at 04:18:30AM +0200, Contact wrote:
> 
> I'm trying to create a multiboot medium using Easy2Boot on a 64 GB USB
> key. Easy2Boot [1] is a "software" (well a collection of files really)
> that relies on grub4dos [2] to work and thus requires payload files to
> be strictly contiguous to do its trick. It's supposed to be set up once
> and then one just has to add files to boot from to a folder to modify
> the multiboot setup.
> 
> The ISO files I have are between 230 MB and 2 GB each. I divided my USB
> drive into two partitions : first one, 16 GB ntfs formatted so that
> grub4dos actually works (for some reasons it doesn't with ext4, plus I
> want to be able to use it with other OS); second one 48 GB formatted as
> ext4 with e2fsprogs version 1.42.9 from Archlinux. However when I try
> to boot from those files I systematically get the "file not contiguous"
> error in Easy2Boot.
> 
> Question is: is it possible to make huge files strictly contiguous on
> ex4, and if yes how?

Most of the time, files don't have to be "strictly contiguous", so
that's not something that we've spent a lot of time trying to achieve.

There is a way to do this if you are willing to use the tip of the
e2fsprogs "maint" branch, and then you put something like this into
your mke2fs.conf file:

	easy2boot = {
		features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
		hash_alg = half_md4
		num_backup_sb = 0
		packed_meta_blocks = 1
		make_hugefiles = 1
		inode_ratio = 4194304
		hugefiles_dir = /
		hugefiles_name = my-big-file
		hugefiles_digits = 0
		hugefiles_size = 16G
		num_hugefiles = 1
		zero_hugefiles = false
	}

Then "mke2fs -T easy2boot /dev/sdc1" will create an ext4 filesystem
with a file called /my-big-file which is guaranteed to be contiguous.

For your particular use case, where you want to create a new
filesystem when you want to create your strictly contiguous file, this
might be the best way to go.

Cheers,

						- Ted

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html