linux-ext4 - Re: Atomic non-durable file write API

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 26 Dec 2010 18:27:34 +0200
From:	Boaz Harrosh <bharrosh@...asas.com>
To:	Olaf van der Spek <olafvdspek@...il.com>
CC:	Nick Piggin <npiggin@...il.com>, Ted Ts'o <tytso@....edu>,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	linux-ext4@...r.kernel.org
Subject: Re: Atomic non-durable file write API

On 12/26/2010 06:02 PM, Olaf van der Spek wrote:
> On Sun, Dec 26, 2010 at 4:55 PM, Boaz Harrosh <bharrosh@...asas.com> wrote:
>> What if you use a soft link? wouldn't that solve all of your problems?
>>
>> - do your fsync/fdatasync of choice in a *backend thread* then at the return
>> - point set to the new link, fsync the link it's very small, therefore fast.
>> - Then delete the old source file.
>>
>> You need a simple "name-version" schema and the "name" is kept soft linked.
>> (You might even skip the last step above and implement an undo stack, some
>>  background management caps on history size)
>>
>>>>
>>>>> and this way has other
>>>>> issues, like losing file meta-data.
>>>>
>>
>> With soft links this is persevered?
>>
>> Same system can be used with lots of files. where the final switch is
>> the set of a single soft-link say to a folder of related files.
> 
> Are you proposing to turn every single file into a symlink?

Sure, a symlink and a "versioned" file for every object. Something similar
to the silly rename of nfs.

Even if you have 1000 files that need the same atomicity treatment
that's not that bad. You should be able to devise a namespace policy
that makes all this nit and tidy.

> How would that solve the meta-data issue?
> 

That's what I asked. Do you want to preserve the original's file
metat-data, or the meta-data of the owner of the new content?
In the first case you'll need a metat-data copy like tar is
using.

> Olaf

The point is to fsync/fdatasync on a background thread and continue
from there where the application is free to go on to the next step.
As if you had a notification when the commit was done (in the background).
So you make it an async pipeline model. The version-naming schem is so the
pipeline can get arbitrary big.

Boaz
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html