linux-kernel - Re: Ext4 and the "30 second window of death"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <49D572E5.4040306@redhat.com>
Date:	Thu, 02 Apr 2009 22:22:29 -0400
From:	Ric Wheeler <rwheeler@...hat.com>
To:	Nick Piggin <nickpiggin@...oo.com.au>
CC:	Matthew Garrett <mjg59@...f.ucam.org>, david@...g.hm,
	Theodore Tso <tytso@....edu>,
	Sitsofe Wheeler <sitsofe@...oo.com>,
	"Andreas T.Auer" <andreas.t.auer_lkml_73537@...us.ath.cx>,
	Alberto Gonzalez <info@...bu.es>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: Ext4 and the "30 second window of death"

Nick Piggin wrote:
> On Friday 03 April 2009 05:38:34 Matthew Garrett wrote:
>   
>> On Fri, Apr 03, 2009 at 05:34:59AM +1100, Nick Piggin wrote:
>>
>>     
>>> Shouldn't applications have a mode to avoid spinning up the disk if it is
>>> so important?
>>>       
>> They do. It's called "Don't use fsync() unless your data needs to be on 
>> disk". I'm not sure why you'd ever want an application to be in anything 
>> but this mode.
>>
>>     
>
> Well you might decide you are willing to sacrifice timely storage of
> logs, or reducing backups in your editor or something. But obviously
> the kernel can't decide which of those fsyncs is safe to omit (or
> turn into a barrier) while staying within the advertised semantics of
> the app. Application obviously can.
>
>   
One thing that you can do at the application level is to try and batch 
up your fsync() requests -  running one fsync (especially on the most 
recently written file) can take down the earlier files with it.

Clearly, this does require some application level complexity, but you 
get the same strong fsync() semantics that you are used to and can run 
almost at non-fsync speeds if the batch size is large enough. Your 
application should not acknowledge it has safely stored any of the files 
locally until it has done an fsync on that particular file.

This technique would work great for an application like rsync, tar, etc. 
For a mail client, you would see a benefit only when you were pulling 
down batches of messages which clearly is a common case if you are still 
reading this thread :-)

The fs_mark program I wrote plays around with the various ways to do 
this if someone is interested in playing around a bit,

Ric

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/