linux-ext4 - Re: fsync() errors is unsafe and risks data loss

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180412215319.jvltbf32gbobrzvz@alap3.anarazel.de>
Date:   Thu, 12 Apr 2018 14:53:19 -0700
From:   Andres Freund <andres@...razel.de>
To:     Jeff Layton <jlayton@...hat.com>
Cc:     Matthew Wilcox <willy@...radead.org>,
        Andreas Dilger <adilger@...ger.ca>,
        20180410184356.GD3563@...nk.org,
        "Theodore Y. Ts'o" <tytso@....edu>,
        Ext4 Developers List <linux-ext4@...r.kernel.org>,
        Linux FS Devel <linux-fsdevel@...r.kernel.org>,
        "Joshua D. Drake" <jd@...mandprompt.com>
Subject: Re: fsync() errors is unsafe and risks data loss

On 2018-04-12 17:27:54 -0400, Jeff Layton wrote:
> On Thu, 2018-04-12 at 13:24 -0700, Andres Freund wrote:
> > At the very least *some* way to *know* that such a failure occurred from
> > userland without having to parse the kernel log. As far as I understand,
> > neither sync(2) (and thus sync(1)) nor syncfs(2) is guaranteed to report
> > an error if it was encountered by writeback in the background.
> > 
> > If that's indeed true for syncfs(2), even if the fd has been opened
> > before (which I can see how it could happen from an implementation POV,
> > nothing would associate a random FD with failures on different files),
> > it's really impossible to detect this stuff from userland without text
> > parsing.
> > 
> 
> syncfs could use some work.

It's really too bad that it doesn't have a flags argument.


> We probably also need to consider how to better track metadata
> writeback errors (on e.g. ext2). We don't really do that properly at
> quite yet either.
> 
> 
> > Even if it'd were just a perf-fs /sys/$something file that'd return the
> > current count of unreported errors in a filesystem independent way, it'd
> > be better than what we have right now.
> > 
> > 1) figure out /sys/$whatnot $directory belongs to
> > 2) oldcount=$(cat /sys/$whatnot/unreported_errors)
> > 3) filesystem operations in $directory
> > 4) sync;sync;
> > 5) newcount=$(cat /sys/$whatnot/unreported_errors)
> > 6) test "$oldcount" -eq "$newcount" || die-with-horrible-message
> > 
> > Isn't beautiful to script, but it's also not absolutely terrible.

ext4 seems to have something roughly like that
(/sys/fs/ext4/$dev/errors_count), and by my reading it already seems to
be incremented from the necessary places.  By my reading XFS doesn't
seem to have something similar.

Wouldn't be bad to standardize...

Greetings,

Andres Freund