linux-ext4 - Re: Is it safe to use the bigalloc feature in the case of ext4 filesystem?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <1aa38330-eaed-ab37-b941-203d970a7727@gmail.com>
Date:   Fri, 30 Jul 2021 15:57:49 +0200
From:   Mikhail Morfikov <mmorfikov@...il.com>
To:     Theodore Ts'o <tytso@....edu>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Is it safe to use the bigalloc feature in the case of ext4
 filesystem?

I have a question concerning the *hugefiles* parameters. When a 
filesystems is created using the hugefiles stanza, it also creates 
lots of chunk-* files inside of the /storage/ dir. You say that it 
guarantees the huge files to be 100% contiguous. But if I create 
the filesystem with the preallocated files that consume the whole 
drive, how am I suppose to use that drive? :) Are the files only
temporary, and should they be removed once the filesystem is 
"ready"? What's the purpose of such options? Do they affect the 
EXT4 metadata in some way? I mean, what's the change compared to 
not using the options?

Also I have a question concerning the hugefiles stanza 
itself -- it's missing the bigalloc feature, should it be there?

On 29/07/2021 19.59, Theodore Ts'o wrote:
>...
> If you see the hugefile and hugefiles stanzas below, that's an example
> of one way bigalloc has gotten a fair amount of use.  In this use case
> mke2fs has pre-allocated the huge data files guaranteeing that they
> will be 100% contiguous.  We're using a 32k cluster becuase there are
> some metadata files where better allocation efficiencies is desired.
> 
>...
>
> 	hugefiles = {
> 		features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
> 		hash_alg = half_md4
> 		reserved_ratio = 0.0
> 		num_backup_sb = 0
> 		packed_meta_blocks = 1
> 		make_hugefiles = 1
> 		inode_ratio = 4194304
> 		hugefiles_dir = /storage
> 		hugefiles_name = chunk-
> 		hugefiles_digits = 5
> 		hugefiles_size = 4G
> 		hugefiles_align = 256M
> 		hugefiles_align_disk = true
> 		zero_hugefiles = false
> 		flex_bg_size = 262144
> 	}
> 
> 	hugefile = {
> 		features = extent,huge_file,bigalloc,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
> 		cluster_size = 32768
> 		hash_alg = half_md4
> 		reserved_ratio = 0.0
> 		num_backup_sb = 0
> 		packed_meta_blocks = 1
> 		make_hugefiles = 1
> 		inode_ratio = 4194304
> 		hugefiles_dir = /storage
> 		hugefiles_name = huge-file
> 		hugefiles_digits = 0
> 		hugefiles_size = 0
> 		hugefiles_align = 256M
> 		hugefiles_align_disk = true
> 		num_hugefiles = 1
> 		zero_hugefiles = false
> 	}