lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 11 Apr 2018 20:02:48 -0700
From:   Matthew Wilcox <willy@...radead.org>
To:     Andres Freund <andres@...razel.de>
Cc:     Andreas Dilger <adilger@...ger.ca>,
        20180410184356.GD3563@...nk.org,
        "Theodore Y. Ts'o" <tytso@....edu>,
        Ext4 Developers List <linux-ext4@...r.kernel.org>,
        Linux FS Devel <linux-fsdevel@...r.kernel.org>,
        Jeff Layton <jlayton@...hat.com>,
        "Joshua D. Drake" <jd@...mandprompt.com>
Subject: Re: fsync() errors is unsafe and risks data loss

On Wed, Apr 11, 2018 at 07:17:52PM -0700, Andres Freund wrote:
> > > While there's some differing opinions on the referenced postgres thread,
> > > the fundamental problem isn't so much that a retry won't fix the
> > > problem, it's that we might NEVER see the failure.  If writeback happens
> > > in the background, encounters an error, undirties the buffer, we will
> > > happily carry on because we've never seen that. That's when we're
> > > majorly screwed.
> > 
> > 
> > I think there are two issues here - "fsync() on an fd that was just opened"
> > and "persistent error state (without keeping dirty pages in memory)".
> > 
> > If there is background data writeback *without an open file descriptor*,
> > there is no mechanism for the kernel to return an error to any application
> > which may exist, or may not ever come back.
> 
> And that's *horrible*. If I cp a file, and writeback fails in the
> background, and I then cat that file before restarting, I should be able
> to see that that failed. Instead of returning something bogus.

At the moment, when we open a file, we sample the current state of the
writeback error and only report new errors.  We could set it to zero
instead, and report the most recent error as soon as anything happens
which would report an error.  That way err = close(open("file")); would
report the most recent error.

That's not going to be persistent across the data structure for that inode
being removed from memory; we'd need filesystem support for persisting
that.  But maybe it's "good enough" to only support it for recent files.

Jeff, what do you think?

Powered by blists - more mailing lists