linux-ext4 - Possible ext4 data corruption with large files and async I/O

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <4B62F688.70404@vectorwise.com>
Date:	Fri, 29 Jan 2010 15:54:00 +0100
From:	Giel de Nijs <giel@...torwise.com>
To:	<linux-ext4@...r.kernel.org>
Subject: Possible ext4 data corruption with large files and async I/O

Dear ext4 devs,

Today I hit a situation where seemingly blocks did not get written to 
disk. I've narrowed it down to the following test case.

Running Fedora Core 12 with kernel 2.6.31.9-174.fc12.x86_64, both on an 
i7 920 and a Core2 Q6600, I executed the following steps:

- create a file
- with kernel async i/o, write a 512kb (haven't tried other sizes) block 
to an offset >4GB, effectively creating a large sparse file
- again with async i/o, write a 512kb block to an offset smaller than 
the previous write, but >4GB
- wait for the kernel async i/o to tell you the writes have succeeded

Now, looking at the file, the second write never seems to have happened. 
When doing this on the same machines on ext3, the behavior is as expected.

As far as I can tell (from the bigger program that triggered this), all 
writes >4GB but < EOF to a sparse file with async i/o aren't executed. 
When creating a large file first (i.e., with dd), everything does work 
as expected.

Attached is some C code that triggers this bug for me.

If you need more information or want me to test some more things, please 
do ask.

Thanks,
Giel de Nijs
VectorWise

View attachment "ext4_bug_2.c" of type "text/x-csrc" (3681 bytes)