Message-ID: <alpine.DEB.2.02.1105301058030.5766@asgard.lang.hm>
Date:	Mon, 30 May 2011 11:02:05 -0700 (PDT)
From:	david@...g.hm
To:	"D. Jansen" <d.g.jansen@...glemail.com>
cc:	Theodore Tso <tytso@....edu>, Oliver Neukum <oneukum@...e.de>,
	akpm@...ux-foundation.org, linux-kernel@...r.kernel.org,
	Dave Chinner <david@...morbit.com>, njs@...ox.com,
	bart@...wel.tk
Subject: Re: [rfc] Ignore Fsync Calls in Laptop_Mode

On Mon, 30 May 2011, D. Jansen wrote:

> On Mon, May 30, 2011 at 3:53 AM,  <david@...g.hm> wrote:
>> On Sun, 29 May 2011, D. Jansen wrote:
>>> On Fri, May 27, 2011 at 4:17 PM, Theodore Tso <tytso@....edu> wrote:
>>>> On May 27, 2011, at 3:12 AM, D. Jansen wrote:
>>>>> That reordering is exactly what I'm talking about. It wasn't my idea.
>>>>> But if I understood it correctly, it's possible that the kernel
>>>>> commits writes of an application, _to one and the same file_, in a
>>>>> non-FIFO order, if the application does not fsync. And this _afaiu_
>>>>> could result in the loss not only of new data, but complete corruption
>>>>> of previously existing data in laptop mode without fsync.
>>>>
>>>> No, you're not understanding the problem.   All layers of the storage
>>>> stack -- including the hard drive -- is allowed to reorder writes.  So
>>>> even if the kernel sends data to the disk in the exact same order that
>>>> the application wrote it, it could still get written in a different
>>>> order,
>>>> because the hard drive itself can reorder writes.   This is necessary
>>>> for performance; if you didn't have this, the storage stack would be
>>>> dog slow, and would consume even more power.
>>>>
>>>> So at least level, the only thing you can count upon is that if you want
>>>> to make sure everything is flushed to stable store, you need to send
>>>> an fsync() command at the application to file system level, or a barrier
>>>> or flush command at the OS to hard drive level.
>>> (...)
>>>> Ordering doesn't matter, because nothing, including the hard drive,
>>>> guarantees ordering.  What does matter is that the fsync() commands
>>>> act like barriers; writes before the fsync() command are guaranteed
>>>> to be written to the disk, and survive a reboot, before any writes after
>>>> the fsync() are processed.  See?
>>>
>>> Ok, thanks a lot! I understand a lot better now!
>>> So we can't live without the fsyncs.
>>>
>>> So what if we would queue the fsyncs along with the writes - we would
>>> just fsync later instead of immediately, in between the writes as they
>>> came in. Then by design previous data could not be corrupted, right?
>>> We would do exactly the same thing, just later.
>>> It'd be kind of a disk write time distortion field.
>>
>> the problem is that the spec for fsync says that your program stops until
>> fsync finishes. If you don't do that then you will corrupt and loose data.
>>
>> so if you delay fsync you will have your application (or desktop manager)
>> freeze until the fsync completes.
>
> So that would not be an option. Freezing until the end of the write
> window is not what we want.
> Neither is ignoring the fsync because that could corrupt data, esp. in
> databases like sqlite.
>>
>> if what you are wanting is the ability to say 'these things must be written
>> before these other things to keep them from being corrupted, but I don't
>> care when they get written (or if they get lost in a crash)' then what you
>> want isn't fsync, it's a barrier.
>
> That sounds great!
> So an fsync call in laptop mode could be interpreted as a barrier
> and we would be reasonably save from corrupting old existing data?

no, you cannot just change a fsync to a barrier, in some cases the data 
absolutly needs to be saved, not just ordered (remember the example of a 
mail server telling the other system that the data can be deleted after a 
fsync returns)

David Lang

Powered by blists - more mailing lists