linux-kernel - Re: [4.1-rc] File was modified, but mtime stayed the same (according to unison)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150609153429.GA704@amd>
Date:	Tue, 9 Jun 2015 17:34:29 +0200
From:	Pavel Machek <pavel@....cz>
To:	Theodore Ts'o <tytso@....edu>, adilger.kernel@...ger.ca,
	linux-ext4@...r.kernel.org,
	kernel list <linux-kernel@...r.kernel.org>, jack@...e.cz
Subject: Re: [4.1-rc] File was modified, but mtime stayed the same (according
 to unison)

On Tue 2015-06-09 11:12:09, Theodore Ts'o wrote:
> On Tue, Jun 09, 2015 at 12:43:30PM +0200, Pavel Machek wrote:
> > 
> > Hi!
> > 
> > Today, I got strange warning from unison:
> > 
> > pavel/.config/chromium/Default/Extension State/LOG.old — transport
> > failure
> > • The source file /data/pavel/.config/chromium/Default/Extension
> > State/LOG.old
> > has been modified but the fast update detection mechanism
> > failed to detect it.  Try running once with the fastcheck
> > option set to 'no'.
> 
> What does this mean, precisely?  Is Unison checking that files have
> been modified using some kind of a checksum or file comparison
> mechanism?  And I assume that the "fast update detection mechanism"
> using mtime?

I believe it is using checksum in the process, yes. 

> And if it has modified, how was it modified (can you do a diff with
> what the other side of the synchronization setup had for that file),
> and do you know by which process. and what was it trying to do?  And
> how is unison being run?

No, sorry, I don't think I can get old version of file.

> One thing that could be going on is that if you have a file which is
> mmap'ed, the mtime field is set the first time the page is modified
> (when the page table entry is set to read/write from read-only).  If
> unison then takes a snapshot of the file, and then file is
> subsequently modified via a write to the mmap'ed page, the mtime field
> will not be updated again.  We *could* constantly reset the page table
> flags but it would be disastrous from a performance standpoint, and if
> mmap is involved, Posix does *not* guarantee that mtime field will be
> set each time a process writes to the mmap'ed segment --- because that
> would be insane.

Ok, I guess mmap() can explain this. So... basically mtime is useless
in detecting if file have been updated?

Thats... not welcome.

I see that constantly updating on-disk timestamp is not
feasible. Could we do something like

  on page_being_mmapped_rw:
	     file.mtime = "future".

  on last_rw_mmap_disappearing:
  	     file.mtime = now().

  stat():
	if file.mtime != "future":
		result.mtime = file.mtime
	else:
		result.mtime = now()

? I see making stat slower is not welcome, but having to read complete
files to determine if they were modified is even worse than that...

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/