linux-ext4 - Re: >16TB issues

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4A660F81.5000509@redhat.com>
Date:	Tue, 21 Jul 2009 13:57:05 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Justin Maggard <jmaggard10@...il.com>
CC:	Andreas Dilger <adilger@....com>, linux-ext4@...r.kernel.org
Subject: Re: >16TB issues

Justin Maggard wrote:
> On Tue, Jul 21, 2009 at 9:10 AM, Andreas Dilger<adilger@....com> wrote:
>>> Error reading block 576192512 (Attempt to read block from filesystem
>>> resulted in short read) while reading inode and block bitmaps.  Ignore
>>> error? no
>>>
>>> e2fsck: Can't read an block bitmap while retrying to read bitmaps for /dev/md2
>>> e2fsck: aborted
>> What is very strange here is that the block numbers being reported as
>> having read errors are not even beyond the 16TB limit.  Assuming 4kB blocks:
>>
>> 576192512 * 4kB = 2304770048kB = 2198GB
>>
>> Are there error messages in syslog/dmesg when this happens?
> 
> No, no error messages from the kernel.  But your llverdev utility
> ended up showing problems on the device.  After asking around on the
> MD mailing list, that was apparently because of the page cache index
> limit (at the time I was using a 32-bit kernel).
> 
> Switching to a 64-bit kernel allowed me to pass the llverdev test and
> get much further with a very large filesystem, but I'm running into
> other issues now.  I wrote up a very simple script to write 2TB files
> onto the filesystem until the device fills up.  It was able to write
> ~16TB, but after that it ran into some problems.  My kernel log now
> has lots of messages like these:
> EXT4-fs error (device md2): ext4_mb_generate_buddy: EXT4-fs: group
> 163548: 32744 blocks in bitmap, 32768 in gd
> - and -
> EXT4-fs error (device md2): ext4_mb_mark_diskspace_used: Allocating
> block 4294967391 in system zone of 131072 group
> 
> I shouldn't need e2fsprogs to be compiled 64-bit as well, right?
> Currently I've got a 64-bit kernel with 32-bit userspace.

It -should- work but it is probably more bug-prone if "unsigned longs"
still lurk.

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html