linux-ext4 - Re: fsync() errors is unsafe and risks data loss

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 11 Apr 2018 19:51:13 -0700
From:   Andres Freund <andres@...razel.de>
To:     Dave Chinner <david@...morbit.com>
Cc:     Andreas Dilger <adilger@...ger.ca>,
        20180410184356.GD3563@...nk.org,
        "Theodore Y. Ts'o" <tytso@....edu>,
        Ext4 Developers List <linux-ext4@...r.kernel.org>,
        Linux FS Devel <linux-fsdevel@...r.kernel.org>,
        Jeff Layton <jlayton@...hat.com>,
        "Joshua D. Drake" <jd@...mandprompt.com>
Subject: Re: fsync() errors is unsafe and risks data loss

Hi,

On 2018-04-11 19:32:21 -0700, Andres Freund wrote:
> And there's cases where that just doesn't help at all. Being able to
> untar a database from backup / archive / timetravel / whatnot, and then
> fsyncing the directory tree to make sure it's actually safe, is really
> not an insane idea.  Or even just cp -r ing it, and then starting up a
> copy of the database.  What you're saying is that none of that is doable
> in a safe way, unless you use special-case DIO using tooling for the
> whole operation (or at least tools that fsync carefully without ever
> closing a fd, which certainly isn't the case for cp et al).

And before somebody argues that that's a too small window to trigger the
problem realistically: Restoring large databases happens pretty commonly
(for new replicas, testcases, or actual fatal issues), takes time, and
it's where a lot of storage is actually written to for the first time in
a while, so it's far from unlikely to trigger bad block errors or such.

Greetings,

Andres Freund