linux-ext4 - Re: [ext3][kernels >= 2.6.20.7 at least] KDE going comatose when FS is under heavy write load (massive starvation)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <46C55E10.900@clusterfs.com>
Date:	Fri, 17 Aug 2007 12:36:32 +0400
From:	Alex Tomas <alex@...sterfs.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
CC:	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: [ext3][kernels >= 2.6.20.7 at least] KDE going comatose when
 FS is under heavy write load (massive starvation)

Andrew Morton wrote:
> Sort-of.  But the per-superpblock, per-inode writeback code is pretty
> careful to avoid livelocks.  The per-inode writeback is a strict single
> linear sweep across the file.  It'll basically write out anything which was
> dirty when it was called.  The per-superblock inode walk isn't as accurate
> as that, becuase of the difficulties of juggling list_heads.  But we're
> slowly working on that, and I suspect it'll be ggod enough for ext3
> purposes already.

I'd say that these are two different mechanism solving different problems:
1) VFS/MM does periodic updates and uses regular writeback
2) data=ordered is to avoid metadata pointing to not-written-yet data

we can't use regular writeback in commit thread as long as it can fall into
allocation. so, we'd have to add one more WB mode (btw, i have a patch which
skips non-allocated blocks in writeback if special WB mode is requested).

OTOH, the faster we go through data sync part of commit, the better. given
that lots of inodes can be dirty with no data to sync, it's going to take
long in some cases. it's especially bad because commit doesn't scale to many
CPUs.

also, why would we need to flush *everything* every 5s? just because ext3 does
this? sounds strange. if somebody really need this we could add this possibility
to regular writeback path (making it tunable). but I'd rather prefer to have
a separate (fast, lightweight, scalable) mechanism to support data=ordered.

thanks, Alex

-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html