linux-ext4 - [Bug 15910] zero-length files and performance degradation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201005101452.o4AEqGeA001448@demeter.kernel.org>
Date:	Mon, 10 May 2010 14:52:16 GMT
From:	bugzilla-daemon@...zilla.kernel.org
To:	linux-ext4@...r.kernel.org
Subject: [Bug 15910] zero-length files and performance degradation

https://bugzilla.kernel.org/show_bug.cgi?id=15910





--- Comment #6 from Peng Tao <bergwolf@...il.com>  2010-05-10 14:24:44 ---
On Mon, May 10, 2010 at 10:56 AM,  <tytso@....edu> wrote:
>> The problem is, dpkg needs to guarantee the system is always usable,
>> and when a crash occurs, say when it's unpacking libc, it's not
>> acceptable for dpkg not to fsync() before rename() as it might end
>> up with an empty libc.so file, even if it might have marked the
>> package as correctly unpacked (wrongly but unknowingly as there's no
>> guarantees), which is not true until the changes have been fully
>> committed to the file system.
>
> Why not unpack all of the files as "foo.XXXXXX" (where XXXXXX is a
> mkstemp filename template) do a sync call (which in Linux is
> synchronous and won't return until all the files have been written),
> and only then, rename the files?  That's going to be the most fastest
> and most efficient way to guarantee safety under Linux; the downside
> is that you need to have enough free space to store the old and the
> new files in the package simultaneously.  But this also is a win,
> because it means you don't actually start overwriting files in a
> package until you know that the package installation is most likely
> going to succeed.  (Well, it could fail in the postinstall script, but
> at least you don't have to worry about disk full errors.)
What about letting fsync() on dir recursively fsync() all
files/sub-dirs in the dir?
Then apps can unpack package in a temp dir, fsync(), and rename.
>
>                                            - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Thanks,
-Bergwolf

--- Comment #7 from Theodore Tso <tytso@....edu>  2010-05-10 14:42:13 ---
On Mon, May 10, 2010 at 10:22:47PM +0800, Peng Tao wrote:
> What about letting fsync() on dir recursively fsync() all
> files/sub-dirs in the dir?
> Then apps can unpack package in a temp dir, fsync(), and rename.

There are programs to do who execute fsync() on a directory, and they
do not expect a recursive fsync() on all files/subdirectories in a
directory.

At least for Linux, sync() is synchronous and will do what you want.
There is unfortunately not a portable way to do what you want short of
fsync'ing all of the files after they are written.  This case is
mostly optimized under ext3/4 (we could do a bit better for ext4, but
the performance shouldn't be disastrous --- certainly much better than
write a file, fsync, rename a file, repeat).

                    - Ted

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html