linux-ext4 - Re: Is it safe to use the bigalloc feature in the case of ext4 filesystem?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 28 Jul 2021 11:36:27 +0200
From:   Mikhail Morfikov <mmorfikov@...il.com>
To:     Theodore Ts'o <tytso@....edu>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Is it safe to use the bigalloc feature in the case of ext4
 filesystem?

Thanks for the answer.

I have one question. Basically there's the /etc/mke2fs.conf file and 
I've created the following stanza in it:

bigdata = {
                errors = remount-ro
                features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize,bigalloc,^uninit_bg,sparse_super2
                inode_size = 256
                inode_ratio = 4194304
                cluster_size = 4M
                reserved_ratio = 0
                lazy_itable_init = 0
                lazy_journal_init = 0
        }

It looks like the cluster_size parameter is ignored in such case (I've 
tried both 4M and 4194304 values), and the filesystem was created with 
64K cluster size (via mkfs -t bigdata -L bigdata /dev/sdb1 ), which is 
the default when the bigalloc feature is set. 

So it looks like the cluster_size doesn't do anything when set in 
/etc/mke2fs.conf . When I used the -C 4M flag (i.e. 
mkfs -t bigdata -L bigdata -C 4M /dev/sdb1), the cluster size was set to 
4M as it should.

Is something wrong with the cluster_size parameter set in the 
/etc/mke2fs.conf file?

----
# mkfs -V
mkfs from util-linux 2.36.1

On 28/07/2021 01.01, Theodore Ts'o wrote:
> On Fri, Jul 23, 2021 at 05:30:13PM +0200, Mikhail Morfikov wrote:
>> In the man ext4(5) we can read the following:
>>
>>     Warning: The bigalloc feature is still under development, 
>>     and may not be fully supported with your kernel or may 
>>     have various bugs. Please see the web page 
>>     http://ext4.wiki.kernel.org/index.php/Bigalloc for details. 
>>     May clash with delayed allocation (see nodelalloc mount 
>>     option).
>>
>> According to the link above, the info is dated back to 2013, 
>> which is a little bit ancient.
>>
>> What's the current status of the feature? Is it safe to use 
>> bigalloc on several TiB hard disks where only big files will be 
>> stored?
> 
> Yes; the places where bigalloc is perhaps not as well tested is
> support FALLOC_FL_COLLAPSE_RANGE, FALLOC_FL_INSERT_RANGE, and
> FALLOC_FL_PUNCH_HOLE.  Bigalloc is also not very efficient for large
> directories (where we allocate a full cluster for each directory
> block).  Older kernels did not handle ENOSPC errors when delayed
> allocation was enabled, but that has since been fixed, and bigalloc is
> passing file system regression tests, so it should safe to use as
> you've described.
> 
> Cheers,
> 
> 					- Ted
> 					
>