linux-kernel - Re: [PATCH 22/43] CacheFiles: Add a hook to write a single page of data to an inode [ver #46]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7093.1238712259@redhat.com>
Date:	Thu, 02 Apr 2009 23:44:19 +0100
From:	David Howells <dhowells@...hat.com>
To:	unlisted-recipients:; (no To-header on input)
Cc:	dhowells@...hat.com, Christoph Hellwig <hch@...radead.org>,
	Nick Piggin <nickpiggin@...oo.com.au>, viro@...iv.linux.org.uk,
	nfsv4@...ux-nfs.org, linux-kernel@...r.kernel.org,
	linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH 22/43] CacheFiles: Add a hook to write a single page of data to an inode [ver #46]

David Howells <dhowells@...hat.com> wrote:

> So, I can apply the attached patch.  It opens a file each time it wants to
> write a page and then closes it again, but it's the best way to avoid the
> ENFILE problem.  I'm benchmarking it at the moment.

Okay.  I'm not sure why, but using ->write() in that patch appears to be _much_
better than using ->write_one_page(), despite the latter doing much less work.

The benchmark was:

	mke2fs -j -O dir_index -b 4096 /dev/sda6
	tune2fs -o user_xattr /dev/sda6
	setenforce 0
	mount /dev/sda6 /var/fscache/ -o user_xattr
	insmod /tmp/sunrpc.ko
	insmod /tmp/lockd.ko
	insmod /tmp/nfs_acl.ko 
	insmod /tmp/auth_rpcgss.ko
	insmod /tmp/fscache.ko
	insmod /tmp/nfs.ko
	insmod /tmp/cachefiles.ko
	sync
	service cachefilesd start
	sync
	mount warthog:/warthog /warthog -o fsc -t nfs4
	date;
		tar cf - /warthog/aaa >/dev/zero &
		tar cf - /warthog/aaa2 >/dev/zero &
		tar cf - /warthog/aaa3 >/dev/zero &
		tar cf - /warthog/aaa >/dev/zero &
		tar cf - /warthog/aaa2 >/dev/zero &
		tar cf - /warthog/aaa3 >/dev/zero &
		tar cf - /warthog/aaa >/dev/zero &
		tar cf - /warthog/aaa2 >/dev/zero;
		wait;
		date
	sync
	echo b >/proc/sysrq-trigger

Where each of /warthog/aaa{,2,3} is about 340MB of kernel tree.  The test box
only has 1GB of RAM.  The server has the whole data set in RAM and is
connected by GigE with more or less no other traffic on the line.


Runs without and with the patch were alternated, and the following results
were produced:

Using write_one_page():

	Thu Apr  2 22:56:45 BST 2009
	Thu Apr  2 23:01:03 BST 2009	4m 18s

	Thu Apr  2 23:11:12 BST 2009
	Thu Apr  2 23:15:12 BST 2009	4m 0s

	Thu Apr  2 23:28:39 BST 2009
	Thu Apr  2 23:32:01 BST 2009	3m 22s

Using write():

	Thu Apr  2 23:05:19 BST 2009
	Thu Apr  2 23:07:31 BST 2009	2m 12s

	Thu Apr  2 23:18:28 BST 2009
	Thu Apr  2 23:20:18 BST 2009	1m 50s

	Thu Apr  2 23:22:52 BST 2009
	Thu Apr  2 23:25:24 BST 2009	2m 32s	

So the patch improves performance by about 2x, it would appear.  The disk
holding the backing fs was much more continually on with the patch, so I think
write() must be promoting some disk I/O.

Or maybe, fput() calling release() is the difference.  Any thoughts?

David
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/