linux-ext4 - Re: [PATCH RFC] ext3 data=guarded v3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LFD.2.00.0904161129460.4042@localhost.localdomain>
Date:	Thu, 16 Apr 2009 11:37:32 -0700 (PDT)
From:	Linus Torvalds <torvalds@...ux-foundation.org>
To:	Chris Mason <chris.mason@...cle.com>
cc:	Mike Galbraith <efault@....de>, Jan Kara <jack@...e.cz>,
	"Theodore Ts'o" <tytso@....edu>,
	Linux Kernel Developers List <linux-kernel@...r.kernel.org>,
	Ext4 Developers List <linux-ext4@...r.kernel.org>
Subject: Re: [PATCH RFC] ext3 data=guarded v3

On Thu, 16 Apr 2009, Chris Mason wrote:
> 
> Ah ok, it is just a missed i_size update.  Basically because file_write
> doesn't wait for page writeback to finish, someone can be updating
> i_size at the same time the end_io handler for the last page is running.
> 
> Git triggers this when it does the sha1flush just before closing the
> file.

Can you say exactly what the IO pattern is?

One of the original git design issues was to actually never _ever_ do 
anything even half-way strange in the filesystem patterns, exactly because 
I've seen so many filesystem bugs over the years.

Now, it turns ou that "original design intent" and "actual code" then 
don't always match, and git did some things that are unusual and triggered 
bugs.

Example: in order to be extra safe, git does "fchown()" after doing all 
the writes to file descriptor just before closing it. I wanted git to make 
it hard to corrupt things by mistake, and marking all the files that only 
get written once (which is most of them) read-only as soon as possible 
seemed to be a great safety feature.

Except, in the process it triggers a network filesystem bug where earlier 
writes were still writeback cached data hadn't made it to the server yet, 
and then the client would do the whole "mark it read-only" before the 
writes had even been done. Oops.

We had a few other issues with just renaming files around (basic rule: 
only rename files _within_ one directory if you want to avoid filesystem 
bugs) and with using "pread/pwrite" (basic rule: pread/pwrite is unusual, 
and is apparently buggy on some operating systems. So avoid them).

Anyway, what was the exact pattern that caused this to show, and maybe I 
can find yet another place where git could just be even more anally safe 
by not doing anything half-way odd?

			Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html