linux-ext4 - Re: xfstests 252 failure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 14 Jun 2011 11:06:46 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Allison Henderson <achender@...ux.vnet.ibm.com>
CC:	xfs-oss <xfs@....sgi.com>,
	Ext4 Developers List <linux-ext4@...r.kernel.org>,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	Josef Bacik <josef@...hat.com>
Subject: Re: xfstests 252 failure

On 6/14/11 10:41 AM, Allison Henderson wrote:
> Hi all,
> 
> I just wanted to get some ideas moving on this question before too
> much time goes by. Ext4 is currently failing xfstest 252, test number
> 12. Currently test 12 is:
> 
>     $XFS_IO_PROG $xfs_io_opt -f -c "truncate 20k" \
>         -c "$alloc_cmd 0 20k" \   
>         -c "pwrite 8k 4k" -c "fsync" \       
>         -c "$zero_cmd 4k 12k" \
>         -c "$map_cmd -v" $testfile | $filter_cmd
>     [ $? -ne 0 ]&&  die_now

so the file should go through these steps:
(H=hole, P=prealloc, D=data)

0k                       20k
|  H |  H |  H |  H |  H | (truncate)
|  P |  P |  P |  P |  P | (alloc_cmd)
|  P |  P |  D |  P |  P | (pwrite)
<fsync>                    (fsync)
|  P |  H |  H |  H |  P | (punch)
 
> and the output is:
> 
>         12. unwritten -> data -> unwritten
> 0: [0..7]: unwritten
> 1: [8..31]: hole
> 2: [32..39]: unwritten
> 
> Ext4 gets data extents here instead of unwritten extents. 

so it's like this?

0: [0..7]: data
1: [8..31]: hole
2: [32..39]: data

> I did some
> investigating and it looks like the fsync command causes the extents
> to be written out before the punch hole operation starts. It looks
> like what happens is that when an unwritten extent gets written to,
> it doesnt always split the extent. If the extent is small enough,
> then it just zeros out the portions that are not written to, and the
> whole extent becomes a written extent. Im not sure if that is
> incorrect or if we need to change the test to not compare the extent
> types.

Yes, it does do that IIRC.

I probably need to look closer, but any test which expects exact
layouts from a filesystem after a series of operations is probably
expecting too much...

>From a data integrity perspective, written zeros is as good as a hole is
as good as preallocated space, so I suppose those should all be acceptable,
though I guess "punch" should result in holes exactly as requested.

> It looks to me that the code in ext4 that does this is supposed to be
> an optimization to help reduce fragmentation. We could change the
> filters to print just "extent" instead of "unwritten" or "data", but
> I realize that probably makes the test a lot less effective for xfs.
> If anyone can think of some more elegant fixes, please let me know.
> Thx!

Josef, what do you think?  It's your test originally.  :)

But if the test is supposed to verify hole punching behavior,
then to me, making unwritten extents equivalent to data extents
full of 0s should be fine.  Perhaps an md5sum step would be 
in order to verify that it is in fact all 0s.

-Eric

> Allison Henderson
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html