linux-ext4 - Re: Atomic non-durable file write API

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 25 Dec 2010 16:24:22 +0100
From:	Olaf van der Spek <olafvdspek@...il.com>
To:	Nick Piggin <npiggin@...il.com>
Cc:	"Ted Ts'o" <tytso@....edu>,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	linux-ext4@...r.kernel.org
Subject: Re: Atomic non-durable file write API

On Sat, Dec 25, 2010 at 12:33 PM, Nick Piggin <npiggin@...il.com> wrote:
>> It's not just about dpkg, I'm still very interested in answers to my
>> original questions.
>
> Arbitrary atomic but non-durable file write operation?

No, not arbitrary writes. It's about complete file writes.
Also, don't forget my question about how to preserve meta-data
including file owner.

> That's significantly
> different to how any part of the pagecache or filesystem or syscall API
> is set up. Writes are not atomic, and syncs are only for durability (not
> atomicity), atomicity is typically built on top of these durable points.
>
> That is quite fundamental functionality and suits simple
> implementations of filesystems and writeback caches.
>
> If you start building complex atomicity semantics, then you get APIs

Atomic semantics are not (that) complex.

> which can't be supported by all filesystems, Linux specific, adds
> complexity from the API through to the pagecache and to the
> filesystems, and is Linux specific.

> Compare that to using cross platform, mature and well tested sqlite
> or bdb, how much reason do we have for implementing such APIs?

Like I said before, it's not about DB-like functionality but about
complete file writes/updates. For example, I've got a file in an
editor and I want to save it.

> It's not that it isn't possible, it's that there is no way we're adding
> such a thing unless it really helps and is going to be widely used.
>
> What exact use case do you have in mind, and what exact API
> semantics do you want, anyway?

Let me copy the original post:
Writing a temp file, fsync, rename is often proposed. However, the
durable aspect of fsync isn't always required and this way has other
issues, like losing file meta-data.
What is the recommended way for atomic non-durable (complete) file writes?

I'm also wondering why FSs commit after open/truncate but before
write/close. AFAIK this isn't necessary and thus suboptimal.

Olaf
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html