linux-kernel - Re: [PATCH RFC] vfs: add a O

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150508024259.GL4327@dastard>
Date:	Fri, 8 May 2015 12:42:59 +1000
From:	Dave Chinner <david@...morbit.com>
To:	Andy Lutomirski <luto@...capital.net>
Cc:	Richard Weinberger <richard.weinberger@...il.com>,
	Zach Brown <zab@...hat.com>,
	Alexander Viro <viro@...iv.linux.org.uk>,
	Sage Weil <sweil@...hat.com>,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	LKML <linux-kernel@...r.kernel.org>,
	"open list:ABI/API" <linux-api@...r.kernel.org>
Subject: Re: [PATCH RFC] vfs: add a O_NOMTIME flag

On Thu, May 07, 2015 at 12:53:46PM -0700, Andy Lutomirski wrote:
> On Thu, May 7, 2015 at 12:09 PM, Richard Weinberger
> <richard.weinberger@...il.com> wrote:
> > On Thu, May 7, 2015 at 7:20 PM, Zach Brown <zab@...hat.com> wrote:
> >> On Thu, May 07, 2015 at 10:26:17AM +1000, Dave Chinner wrote:
> >>> On Wed, May 06, 2015 at 03:00:12PM -0700, Zach Brown wrote:
> >>> > Add the O_NOMTIME flag which prevents mtime from being updated which can
> >>> > greatly reduce the IO overhead of writes to allocated and initialized
> >>> > regions of files.
> >>>
> >>> Hmmm. How do backup programs now work out if the file has changed
> >>> and hence needs copying again? ie. applications using this will
> >>> break other critical infrastructure in subtle ways.
> >>
> >> By using backup infrastructure that doesn't use cmtime.  Like btrfs
> >> send/recv.  Or application level backups that know how to do
> >> incrementals from metadata in giant database files, say, without
> >> walking, comparing, and copying the entire thing.
> >
> > But how can Joey random user know that some of his
> > applications are using O_NOMTIME and his KISS backup
> > program does no longer function as expected?
> >
> 
> Joey random user can't have a working KISS backup anyway, though,
> because we screw up mtime updates on mmap writes.  I have patches
> gathering dust that fix that, though.

They are close enough to be good for backup purposes. The mtime only
need change once per backup period - it doesn't need to be
millisecond accurate. Yes, I know you needed that changed for
different reasons (avoid variable page fault latency), but it
doesn't matter for once-a-day or even once-an-hour incremental
backup scans.

Besides, anyone who cares about accurate backups is doing a backup
from a snapshot so they data and metadata is consistent across the
entire backup. And that makes worries about mmap and mtime
completely irrelevant because a snapshot freezes the filesystem and
hence cleans all the mapped pages. Once the snapshot is taken
the next mmap write will trigger a page fault and so change the
mtime and it will be picked up in the next backup scan...

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/