linux-ext4 - Re: [ext3][kernels >= 2.6.20.7 at least] KDE going comatose when FS is under heavy write load (massive starvation)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070427193130.GD5967@schatzie.adilger.int>
Date:	Fri, 27 Apr 2007 13:31:30 -0600
From:	Andreas Dilger <adilger@...sterfs.com>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	Marat Buharov <marat.buharov@...il.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Mike Galbraith <efault@....de>,
	LKML <linux-kernel@...r.kernel.org>,
	Jens Axboe <jens.axboe@...cle.com>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>,
	Alex Tomas <alex@...sterfs.com>
Subject: Re: [ext3][kernels >= 2.6.20.7 at least] KDE going comatose when FS is under heavy write load (massive starvation)

On Apr 27, 2007  08:30 -0700, Linus Torvalds wrote:
> On a good filesystem, when you do "fsync()" on a file, nothing at all 
> happens to any other files. On ext3, it seems to sync the global journal, 
> which means that just about *everything* that writes even a single byte 
> (well, at least anything journalled, which would be all the normal 
> directory ops etc) to disk will just *stop* dead cold!
> 
> It's horrid. And it really is ext3, not "fsync()".
> 
> I used to run reiserfs, and it had its problems, but this was the 
> "feature" of ext3 that I've disliked most. If you run a MUA with local 
> mail, it will do fsync's for most things, and things really hickup if you 
> are doing some other writes at the same time. In contrast, with reiser, if 
> you did a big untar or some other big write, if somebody fsync'ed a small 
> file, it wasn't even a blip on the radar - the fsync would sync just that 
> small thing.

It's true that this is a "feature" of ext3 with data=ordered (the default),
but I suspect the same thing is now true in reiserfs too.  The reason is
that if a journal commit doesn't flush the data as well then a crash will
result in garbage (from old deleted files) being visible in the newly
allocated file.  People used to complain about this with reiserfs all the
time having corrupt data in new files after a crash, which is why I believe
it was fixed.

There definitely are some problems with the ext3 journal commit though.
If the journal is full it will cause the whole journal to checkpoint out
to the filesystem synchronously even if just space for a small transaction
is needed.  That is doubly bad if you have a very large journal.  I believe
Alex has a patch to have it checkpoint much smaller chunks to the fs.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html