linux-kernel - Summary Re: [rfc] Ignore Fsync Calls in Laptop

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:	Sat, 11 Jun 2011 14:24:58 +0200
From:	"D. Jansen" <d.g.jansen@...glemail.com>
To:	"Ted Ts'o" <tytso@....edu>,
	"D. Jansen" <d.g.jansen@...glemail.com>, david@...g.hm,
	Oliver Neukum <oneukum@...e.de>, akpm@...ux-foundation.org,
	linux-kernel@...r.kernel.org, Dave Chinner <david@...morbit.com>,
	njs@...ox.com, bart@...wel.tk,
	Stewart Smith <stewart@...mingspork.com>
Subject: Summary Re: [rfc] Ignore Fsync Calls in Laptop_Mode

It seems unlikely that an automatic workaround for fsync will make it
into kernel for laptop mode, as none of the kernel maintainers seem to
think this is a good idea.

But it seems it makes sense to provide write barrier support for user
space. This would not be a write barrier as the kernel provides it for
kernel space atm
(https://docs.fedoraproject.org/en-US/Fedora/14/html/Storage_Administration_Guide/writebarr.html).

But it would be an interface that lets userspace tell the kernel that
certain writes must be committed before other writes, without the need
for flushing as fsync does. This could provide a benefit especially
for databases.

The idea is that writes, which are dependent on each other in the way
that one write may make a file unreadable or lead to data loss if it
is committed to disk in the wrong order, can be queued. This way the
heavy fsyncs with their corresponding flush can be avoided.

As Stewart wrote:

"Some of them [databases] in some situations could. For those used by
applications to
store data (such as how sqlite is commonly used) - it could be *great* -
especially for laptop mode. The kernel can then flush this all out any
time it wants - preserving barriers of course. If filesystem can do
atomic writes (btrfs?) then it could combine barriers into less overall
work (e.g. write block 1, barrier, repeat 10 times can turn into 1 block
write of block 1).

For database systems that tend to take over the whole system (such as
InnoDB (MySQL and Drizzle), MySQL Cluster, PostgreSQL, Oracle) - the
possibility of using such a barrier call is small. There are good
reasons why database systems manage all their own buffers and flushing -
we have a *LOT* more knowledge about the data than the kernel will ever
have and so we can make much better decisions about what to flush and
when."

On Thu, Jun 2, 2011 at 9:42 AM, D. Jansen <d.g.jansen@...glemail.com> wrote:
>
> On Tue, May 31, 2011 at 4:03 AM, Ted Ts'o <tytso@....edu> wrote:
> > The real fix is that applications need to be fixed to be less
> > write-happy.
>
> So do you agree to export barriers to user space? Because as you said
> yourself, that would be the only way for apps to avoid fsync without
> losing data safety. And since you want the problem fixed with the
> apps, we need to provide them with the means to do so, right?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/