linux-ext4 - e2fsck bug? - Sparse files corrupt after "e2fsck -E bmap2extent".

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 23 May 2017 00:20:59 +0100
From:   Marc Thomas <marc@...gonfly.plus.com>
To:     linux-ext4@...r.kernel.org
Subject: e2fsck bug? - Sparse files corrupt after "e2fsck -E bmap2extent".

Hi All,

I hope this is the correct place for ext4 / e2fsprogs bug reports. I
think this is a bug.

As per the title, I've discovered sparse files appear corrupt after
running "e2fsck -E bmap2extent" on a filesystem I'm migrating.
Tested with kernel.org 4.10.15 and 4.11.1, using e2fsprogs-1.43.4 on an
x86_64 system.

In short, I'm carrying out a data migration from a snapshot of an ext3
filesystem to new storage, then expanding the filesystem, converting (in
place) to ext4 (as per
https://ext4.wiki.kernel.org/index.php/UpgradeToExt4), before
de-fragmenting the new filesystem with e4defrag.

As part of the testing, I created md5sums of all the source files. After
a "dry run" of the migration, a minority of files (approx 30 out of
1095578 inodes) were found to be corrupt (ie the md5sum had changed).

I repeated the process a second time, checking the md5sums after each
step of the migration. It appears the corruption occurs after running
"e2fsck -E bmap2extent -fy" on the newly converted ext4 filesystem.

The one thing I can find in common is all the affected files are sparse
files.

Here's a few example files:

linux-2.6.30/arch/x86/kernel/acpi/realmode/wakeup.bin: FAILED
linux-2.6.30/arch/x86/boot/compressed/vmlinux.bin: FAILED
marc.old/.mozilla/firefox/ecyfs7l9.default/Cache/_CACHE_003_: FAILED
Linux4.x/linux-4.10.12/arch/x86/realmode/rm/realmode.bin: FAILED
Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin: FAILED

They are all definitely sparse files, eg:

$ ls -l Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin
-rwxr-xr-x 1 marc users 21080 Mar  8 12:03
Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin

$ du -k Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin
16      Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin

$ du --apparent -k Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin
21      Linux4.x/linux-4.10.1/arch/x86/realmode/rm/realmode.bin

Would you agree this is a bug? As I understand it, reading from an
"unpopulated" region of a sparse file should return all zeros - so the
md5sum should be the same before and after migration.

I'm happy to provide more details and do testing if required. I have an
offline copy of the source filesystem so can run the migration again.
Let me know if there's anything you need.

Thanks & Kind Regards,
Marc