linux-kernel - Re: Linux 2.6.29 - delayed metadata for delayed allocation?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 27 Mar 2009 13:17:34 +0100
From:	"Andreas T.Auer" <andreas.t.auer_lkml_73537@...us.ath.cx>
To:	Theodore Tso <tytso@....edu>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
CC:	Matthew Garrett <mjg59@...f.ucam.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	David Rees <drees76@...il.com>, Jesper Krogh <jesper@...gh.cc>
Subject: Re: Linux 2.6.29 - delayed metadata for delayed allocation?

On 2009-03-27 at 12:24 Theodore Tso wrote:
> When I was growing up we were trained to *always* check error returns
> from *all* system calls, and to *always* fsync() if it was critical
> that the data survive a crash.
But there are a lot of applications for which the survival of the data
is not this critical as long as the old data is still available.

Data are the important stuff, metadata helps to find them. Even though
there are a lot of cases, where the information is just stored in the
metadata.
If you write metadata for not-yet-existing data to disk, then these are
inconsistent, corrupt, dirty.

Why don't you just delay the writing of these dirty metadata, too, until
they are clean? So nothing is written until the next sync and then

1) write the data to the nicely allocated places.
2) journal the metadata for consistency
3) write the metadata
4) cleanup the journal

That way you can have sophisticated allocation and keep a consistent
filesystem without data loss due to re-ordering.

Clean metadata-changes which don't have delayed data might be
written/journaled immediately. That rises the question, whether dirty
metadata changes should be skipped or whether a dirty metadata change
should block later clean metadata changes to inhibit the re-ordering of
changes. This should be a mount-option IMHO. Keeping the order of
fs-changes has a big advantage in many cases.

Syncing data on renames would decrease your performance which you want
to increase with delayed allocation. Delayed metadata would mostly keep
this performance gain, right?

Andreas

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/