linux-ext4 - Re: ext4 64bit (disk >16TB) question

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87mykirhb0.fsf@frosties.localdomain>
Date:	Tue, 15 Jul 2008 23:03:31 +0200
From:	Goswin von Brederlow <goswin-v-b@....de>
To:	Ric Wheeler <ricwheeler@...il.com>
Cc:	Andreas Dilger <adilger@....com>,
	"Jose R. Santos" <jrs@...ibm.com>,
	Goswin von Brederlow <goswin-v-b@....de>,
	linux-ext4@...r.kernel.org
Subject: Re: ext4 64bit (disk >16TB) question

Ric Wheeler <ricwheeler@...il.com> writes:

> Andreas Dilger wrote:
>> Jose,
>> while waiting for the "efficient bitmap" support, how hard would it be
>> to implement "inefficient bitmaps" that just malloc some GB of memory
>> if needed?  This would at least allow people with huge devices to test
>> mke2fs/ext4/e2fsck in the meantime.
>>
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Sr. Staff Engineer, Lustre Group
>> Sun Microsystems of Canada, Inc.
>>
>>
>
> I think that would be very useful - how much DRAM would we need for a
> 16TB file system ;-) ?
>
> ric

The patched 1.39 e2fsprogs managed to format a 16TIB under kvm with
1GiB ram and 128k swap. A 32TiB disk format uses nearly 1GiB ram for mkfs
alone and eventualy managed to deadlock the I/O layer in kvm with
1.5GB ram and 128k swap. (Something I'm sure is kvms fault. :)

But fsck is suposed to eat more by a factor (see other mails in
thread). So having 4-16GiB ram is probably recommended for anyone
thinking about testing.

I used the sparse_create script linked on one of the ext4 wiki pages
with a sparse loopback file and used mke2fs -i $((64*1024*1024)) to
speed up things. With that a 16TiB ext4 uses somewhat over 4GiB
freshly formated.

MfG
        Goswin
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html