linux-ext4 - Writing large files into a ext4 with mkfs is truncating files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJTo0LYhwc978ywMkZ1v3xZHsW4DVwrszKnPoewvzvNixD_Pcg@mail.gmail.com>
Date:   Fri, 1 Feb 2019 00:14:08 +0000
From:   "Burton, Ross" <ross.burton@...el.com>
To:     linux-ext4 <linux-ext4@...r.kernel.org>
Subject: Writing large files into a ext4 with mkfs is truncating files

Hi,

We're creating file systems using mkfs.ext4 -d /path/to/fs, but I'm
seeing file corruption when adding a large file, which appears to
trigger if the file is over 4GB.

For example, 3GB of random data:
$ mkdir fs
$ dd if=/dev/urandom of=fs/foo status=progress bs=1M count=3K
$ md5sum fs/foo
e3c7e0243208b49b55dd8e4f7681302f  fs/foo
$ truncate -s 6G foo.ext4
$ /sbin/mkfs.ext4  foo.ext4 -d fs/
$ sudo mount foo.ext4 /mnt
$ md5sum /mnt/foo
e3c7e0243208b49b55dd8e4f7681302f  /mnt/foo

But by just changing count=3K to count=5K:

...
a2c04fdb5e717f3b79f6989390e1ce50  fs/foo
e5e54aec74285b3e750fe0b877363e1d  /mnt/foo

A little sleuthing:

$ cmp fs/foo  /mnt/foo
fs/foo /mnt/foo differ: byte 1073741825, line 4194938

Okay that's not the 4GB boundary. But dumping the values 10 bytes
either side of that point:

$ dd if=fs/foo skip=1073741815 bs=1 count=20|od -b -Anone
 333 024 335 134 327 122 306 256 064 107 345 031 037 125 107 355
$ dd if=/mnt/foo skip=1073741815 bs=1 count=20|od -b -Anone
 333 024 335 134 327 122 306 256 064 000 000 000 000 000 000 000

At some point mkfs.ext4 stopped copying content and simply padded with zeros.

This is e2fsprogs 1.43.4 and 1.44.3.  I've not yet tried 1.44.5 or git master.

Ross