linux-ext4 - new ext4 filesystem vs. converted ext3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <87tyc6pov6.fsf@algae.riseup.net>
Date:	Fri, 03 Jun 2011 17:49:17 -0400
From:	Micah Anderson <micah@...eup.net>
To:	linux-ext4@...r.kernel.org
Subject: new ext4 filesystem vs. converted ext3

Hello,

I recently ran into the subdirectory scalability issue of ext3 on our
listserver. Thankfully, ext4 exists, so it was finally time to move to
it. I followed the published process to convert the ext3 filesystem to
ext4[0] and things worked well, I'm no longer hitting the 32000 limit on
subdirectories.

However, ever since I did that change, I've noticed an increase in I/O
wait state on the CPUs. I've been trying to determine why, and if there
were some things I should tune on this ext4 filesystem.

First thing I found was that the Filesystem Features of an ext4
filesystem that were converted from ext3 are different than those that
are set on a filesystem that was created as ext4 (on Debian Squeeze),
From tune2fs -l:

a newly created ext4 has:
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super huge_file  uninit_bg dir_nlink extra_isize

my migrated one has: 
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent sparse_super large_file uninit_bg

The difference being new filesystems have 'flex_bg', 'huge_file' and
'extra_isize' set, and the converted filesystem has 'large_file' and
'uninit_bg'. I'm not sure I understand why the difference and wonder if
someone can explain to me why? Or if I should be changing these?

Currently the filesystem is mounted with just rw,relatime set, and I
wonder if there is a combination of some tune2fs changes, and mount
options that we would benefit from.

Basically this filesystem has a lot of small files, there is a set of
spool directories where there are a lot of small writes, small reads and
deletes. There is also an archive of list postings, which is a fairly
large set of small files, which is mostly small writes, reads every once
and a while. There is also a series of stats that happen on a large
number of subdirectories.

Thank you for any suggestions!
micah

0. https://ext4.wiki.kernel.org/index.php/Ext4_Howto#Converting_an_ext3_filesystem_to_ext4

Content of type "application/pgp-signature" skipped