[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <51B3F68B.6080709@huawei.com>
Date: Sun, 9 Jun 2013 11:29:15 +0800
From: Zhao Hongjiang <zhaohongjiang@...wei.com>
To: "Theodore Ts'o" <tytso@....edu>
CC: <linux-ext4@...r.kernel.org>, <hch@....de>
Subject: Re: xfstests failure generic/239
On 2013/6/9 10:37, Zhao Hongjiang wrote:
> On 2013/6/9 6:30, Theodore Ts'o wrote:
>> On Sat, Jun 08, 2013 at 11:13:35AM +0800, Zhao Hongjiang wrote:
>>>
>>> I run xfstests #239 against mainline 3.10.0-rc3, unfortunately it failure in my QEMU. I run the
>>> case a hundred times, it certainly hit the failure several times. The failure msg is as follow:
>>>
>>> FSTYP -- ext4
>>> PLATFORM -- Linux/x86_64 3.10.0-rc3-mainline
>>>
>>> generic/239 1s ... - output mismatch (see /home/zhj/xfstests/results/generic/239.out.bad)
>>> --- tests/generic/239.out 2013-06-07 22:04:09.000000000 -0400
>>> +++ /home/zff/xfstests/results/generic/239.out.bad 2013-06-07 22:04:09.000000000 -0400
>>> @@ -1,2 +1,515 @@
>>> QA output created by 239
>>> +hostname: Host name lookup failure
>>
>> OK, so this hostname failure is weird; I'm not sure what's causing
>> this, but this I presume unrelated to the failure at hand.
>>
>>> Silence is golden
>>> +0: 0x0
>>> +1: 0x0
>>> +2: 0x0
>>> +3: 0x0
>>
>> This indicates a problem. Test generic/239 is running
>> aio-dio-hole-filling-race.c, which submits an asynchronous, direct I/O
>> 4k write with a buffer containing non-zero contents to a sparse file,
>> and once the I/O has completed, it uses pread to read it back, using
>> the same descriptor, so it is doing the read using direct I/O. It
>> then checks to see if the read returns zero or not.
>>
>> The "XX: 0x0" lines indicates that buffer is zero, which implies that
>> somehow aio_complete() is getting called before the uninitialized to
>> initialized conversion is taking place. I'm not seeing how this is
>> happening, though, so I'm a bit puzzled. If there are any unwritten
>> extents, we don't call aio_complete() in ext4_end_io_dio(), but
>> instead the conversion is queued via a call to ext4_add_compete_io(),
>> and and aio_done() is only called on the iocb after the conversion is
>> complete.
>>
>> Can anyone see something that I might be missing?
>>
>> - Ted
>>
>> P.S. Zhao, what was the hardware that you using to find this failure?
>
> I'am use x86 and start a qumu-kvm to run the test.
>
>> I'm not seeing it, but then again if the failure is only happening
>> once every few hundred runs that might explain it. I'm perhaps
>
> And as Christoph Hellwig said "the race is very easy to hit by using QEMU with
> native AIO support on a sparse image, and the result is filesystem corruption
> in the guest", i also run the test on the host, but nerver see the failure.
Sorry, i run the test on host a hundred times again, there are six failures.
>
>> wondering if we should add a mode to aio-dio-hole-filling-race.c which
>> allows it to try the race a large number of times, instead of just
>> once.
>
> This seems necessary, i'll give a patch for this.
> - Zhao
>>
>> P.P.S. One thought.... perhaps it might be useful to have a debug
>> mode where we use queue_delayed_work() to submit the conversion
>> request to the workqueue. It will of course make certain workloads
>> run slow as molasses, but it might expose some races so we can see
>> them more easily.
>>
>> .
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists