linux-ext4 - Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8739m0g3u7.fsf@rho.meyering.net>
Date:	Sat, 02 Apr 2011 15:50:08 +0200
From:	Jim Meyering <jim@...ering.net>
To:	Pádraig Brady <P@...igBrady.com>
Cc:	8411@...bugs.gnu.org, linux-ext4@...r.kernel.org
Subject: Re: bug#8411: due to missing sync even on 2.6.39, cp fails to copy an odd file

Pádraig Brady wrote:
> On 02/04/11 12:16, Jim Meyering wrote:
>> Hi Pádraig,
>> As of this change,
>>
>>     copy: with fiemap copy, only sync when needed
>>     http://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=f69876e55
>>
>> fiemap copy with extents beyond EOF can fail on ext4 even with
>> Fedora 15 (2.6.38) and rawhide's 2.6.39 kernel.
>>
>> Here we construct an odd file.  First, preallocate 10MB of space,
>> and then write 5KiB of random data into the beginning of that:
>>
>>     $ fallocate -l 10000000 -n k
>>     $ dd count=10 if=/dev/urandom conv=notrunc iflag=fullblock of=k
>>
>> However, when we try to copy "k", we get a file, "k2" of the
>> expected size, but with only NUL bytes for contents:
>
> So the extent info is not updated until sync(),
> which means cp will consider the "unwritten" extent
> as NUL data and not bother to read it :(
>
> I guess this is a corner case that was missed
> in the fixups for ext4 (and btrfs?) in 2.6.38 for this?
> I've copied ext4 devs for clarification.
> I.E. if you do this:
>
>   fallocate -l 10000000 -n k
>   dd count=10 if=/dev/urandom conv=notrunc iflag=fullblock of=k
>   filefrag -v k
>
> Do you get all extents still unwritten.
> I do on my 2.6.35.11-83.fc14.i686 kernel, but I expected that.

Right.  The new extent remains unwritten until after a sync
with ext4 on both Fedora 15 and rawhide kernels.

Here's F15:

    $ fallocate -l 10000000 -n k
    $ dd count=10 if=/dev/urandom conv=notrunc iflag=fullblock of=k
    ...
    $ filefrag -v k
    Filesystem type is: ef53
    File size of k is 5120 (2 blocks, blocksize 4096)
     ext logical physical expected length flags
       0       0  6498304            2442 unwritten,eof
    k: 1 extent found
    $ sync
    $ filefrag -v k
    Filesystem type is: ef53
    File size of k is 5120 (2 blocks, blocksize 4096)
     ext logical physical expected length flags
       0       0  6498304               2 eof
       1       2  6498306            2440 unwritten,eof
    k: 1 extent found

Same problem with rawhide, but slightly different layout:

    $ fallocate -l 10000000 -n k
    $ dd count=10 if=/dev/urandom conv=notrunc iflag=fullblock of=k
    ...
    $ filefrag -v k
    Filesystem type is: ef53
    File size of k is 5120 (2 blocks, blocksize 4096)
     ext logical physical expected length flags
       0       0  2054144            2048 unwritten,eof
       1    2048  2060288  2056191    394 unwritten,eof
    k: 2 extents found
    $ filefrag -v k
    [Exit 1]
    $ sync
    $ filefrag -v k
    Filesystem type is: ef53
    File size of k is 5120 (2 blocks, blocksize 4096)
     ext logical physical expected length flags
       0       0  2054144               2 eof
       1       2  2054146            2046 unwritten,eof
       2    2048  2060288  2056191    394 unwritten,eof
    k: 2 extents found
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html