linux-kernel - Re: Linux 2.6.29

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090330003948.GA13356@mit.edu>
Date:	Sun, 29 Mar 2009 20:39:48 -0400
From:	Theodore Tso <tytso@....edu>
To:	Mark Lord <lkml@....ca>,
	Stefan Richter <stefanr@...6.in-berlin.de>,
	Jeff Garzik <jeff@...zik.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Matthew Garrett <mjg59@...f.ucam.org>,
	Alan Cox <alan@...rguk.ukuu.org.uk>,
	Andrew Morton <akpm@...ux-foundation.org>,
	David Rees <drees76@...il.com>, Jesper Krogh <jesper@...gh.cc>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: Linux 2.6.29

On Mon, Mar 30, 2009 at 10:14:51AM +1100, Dave Chinner wrote:
> This is a clear case where you want metadata changed before data is
> committed to disk. In many cases, you don't even want the data to
> hit the disk here.
> 
> Similarly, rsync does the magic open,write,close,rename sequence
> without an fsync before the rename. And it doesn't need the fsync,
> either. The proposed implicit fsync on rename will kill rsync
> performance, and I think that may make many people unhappy....

I agree.  But unfortunately, I think we're going to be bullied into
data=ordered semantics for the open/write/close/rename sequence, at
least as the default.  Ext4 has a noauto_da_alloc mount option (which
Eric Sandeen suggested we rename to "no_pony" :-), for people who
mostly run sane applications that use fsync().

For people who care about rsync's performance and who assume that they
can always restart rsync if the system crashes while the rsync is
running could, rsync could add Yet Another Rsync Option :-) which
explicitly unlinks the target file before the rename, which would
disable the implicit fsync().

> > Much easier and more reliable to centralize it there, rather than
> > rely (falsely) upon thousands of programs each performing numerous
> > performance-killing fsync's.
> 
> The filesystem should batch the fsyncs efficiently. if the
> filesystem doesn't handle fsync efficiently, then it is a bad
> filesystem choice for that workload....

All I can do is apologize to all other filesystem developers profusely
for ext3's data=ordered semantics; at this point, I very much regret
that we made data=ordered the default for ext3.  But the application
writers vastly outnumber us, and realistically we're not going to be
able to easily roll back eight years of application writers being
trained that fsync() is not necessary, and actually is detrimental for
ext3.

	       		      	       	      - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/