linux-ext4 - 16 TB filesystem limit on 32bit machine

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 20 Feb 2012 17:31:15 +0200
From:	Rabeeh Khoury <rabeeh3000@...il.com>
To:	linux-ext4@...r.kernel.org
Subject: 16 TB filesystem limit on 32bit machine

I'm trying to figure out all issues with regards >16TB filesystem
support on ARM (32bit) machines.
Clearly this issue was hot few years ago, part of the discussions -

https://bugzilla.kernel.org/show_bug.cgi?id=12556
http://www.redhat.com/archives/dm-devel/2009-July/msg00131.html

And there was Eric's patch of checking length of pgoff_t and
accordingly refuse mount.

Now, today with 4TB hard drives in the market, having 5 of those on an
ARM machine is really common and the ext4 limitation is becoming more
reachable and requires attention.
What i'm trying to achieve is the following two items -

---- item #1 ---
Understand where the limitation is really coming from? Is this ext4
implementation limitation or 32bit machines will never work with >16TB
filesystems?
I understand that there is a 16TB file size limitation (2^32*4K page
size so you won't be able to mmap() further than that point) but how
is that related to filesystem size?
Will 64KB page size fix this issue (ARM supports 4KB and 64KB pages) -
clearly memory fragmentation will be a hit here.

----- item #2 ---
Reproduce a failing scenario.
For now i'v created a 24TB volume (thin provisioned) - RAID-0 on a 3 x
loopback on a 3 x truncted 8TB consisting total of 24TB volume
mkfs.ext4 /dev/md0 (e2fsprogrs 1.42 - thanks for the >16TB support)
mount on a hacked kernel (#define pgoff_t unsigned long long thus
making filesystem mounting check disappear)
The volume mounts ok; now how do i get into corruption? I don't have
physical 24TB drive, so best if there is a pin-pointed to test to
reproduce the issue.

Best regards,
Rabeeh
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html