linux-kernel - [PATCH] NFS: Pagecache usage optimization on nfs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <6.0.0.20.2.20090217132810.05709598@172.19.0.2>
Date:	Tue, 17 Feb 2009 13:55:12 +0900
From:	Hisashi Hifumi <hifumi.hisashi@....ntt.co.jp>
To:	Trond.Myklebust@...app.com, linux-nfs@...r.kernel.org
Cc:	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org
Subject: [PATCH] NFS: Pagecache usage optimization on nfs

Hi, Trond.

I wrote "is_partially_uptodate" aops for nfs client named nfs_is_partially_uptodate().
This aops checks that nfs_page is attached to a page and read IO to a page is 
within the range between wb_pgbase and wb_pgbase + wb_bytes of the nfs_page. 
If this aops succeed, we do not have to issue actual read IO to NFS server 
even if a page is not uptodate because the portion we want to read are uptodate.
So with this patch random read/write mixed workloads or random read after random write 
workloads can be optimized and we can get performance improvement.
 
I did benchmark test using sysbench.

sysbench --num-threads=16 --max-requests=100000 --test=fileio --file-block-size=2K 
--file-total-size=200M --file-test-mode=rndrw --file-fsync-freq=0 
--file-rw-ratio=0.5 run

The result was:

-2.6.29-rc4

Operations performed:  33356 Read, 66682 Write, 128 Other = 100166 Total
Read 65.148Mb  Written 130.24Mb  Total transferred 195.39Mb  (3.1093Mb/sec)
 1591.97 Requests/sec executed

Test execution summary:
    total time:                          62.8391s
    total number of events:              100038
    total time taken by event execution: 841.7603
    per-request statistics:
         min:                            0.0000s
         avg:                            0.0084s
         max:                            16.4564s
         approx.  95 percentile:         0.0446s

Threads fairness:
    events (avg/stddev):           6252.3750/306.48
    execution time (avg/stddev):   52.6100/0.38


-2.6.29-rc4 + patch

Operations performed:  33346 Read, 66662 Write, 128 Other = 100136 Total
Read 65.129Mb  Written 130.2Mb  Total transferred 195.33Mb  (5.0113Mb/sec)
 2565.81 Requests/sec executed

Test execution summary:
    total time:                          38.9772s
    total number of events:              100008
    total time taken by event execution: 339.6821
    per-request statistics:
         min:                            0.0000s
         avg:                            0.0034s
         max:                            1.6768s
         approx.  95 percentile:         0.0200s

Threads fairness:
    events (avg/stddev):           6250.5000/302.04
    execution time (avg/stddev):   21.2301/0.45


I/O performance was significantly improved by following patch.
Please merge my patch.
Thanks.

Signed-off-by: Hisashi Hifumi <hifumi.hisashi@....ntt.co.jp>

diff -Nrup linux-2.6.29-rc5.org/fs/nfs/file.c linux-2.6.29-rc5/fs/nfs/file.c
--- linux-2.6.29-rc5.org/fs/nfs/file.c	2009-02-16 12:31:18.000000000 +0900
+++ linux-2.6.29-rc5/fs/nfs/file.c	2009-02-16 13:05:29.000000000 +0900
@@ -449,6 +449,7 @@ const struct address_space_operations nf
 	.releasepage = nfs_release_page,
 	.direct_IO = nfs_direct_IO,
 	.launder_page = nfs_launder_page,
+	.is_partially_uptodate = nfs_is_partially_uptodate,
 };
 
 static int nfs_vm_page_mkwrite(struct vm_area_struct *vma, struct page *page)
diff -Nrup linux-2.6.29-rc5.org/fs/nfs/read.c linux-2.6.29-rc5/fs/nfs/read.c
--- linux-2.6.29-rc5.org/fs/nfs/read.c	2009-02-16 12:31:18.000000000 +0900
+++ linux-2.6.29-rc5/fs/nfs/read.c	2009-02-16 13:05:29.000000000 +0900
@@ -599,6 +599,33 @@ out:
 	return ret;
 }
 
+int nfs_is_partially_uptodate(struct page *page, read_descriptor_t *desc,
+				unsigned long from)
+{
+	struct inode *inode = page->mapping->host;
+	unsigned to;
+	struct nfs_page *req = NULL;
+
+	spin_lock(&inode->i_lock);
+	if (PagePrivate(page)) {
+		req = (struct nfs_page *)page_private(page);
+		if (req)
+			kref_get(&req->wb_kref);
+	}
+	spin_unlock(&inode->i_lock);
+	if (!req)
+		return 0;
+
+	to = min_t(unsigned, PAGE_CACHE_SIZE - from, desc->count);
+	to = from + to;
+	if (from >= req->wb_pgbase && to <= req->wb_pgbase + req->wb_bytes) {
+		nfs_release_request(req);
+		return 1;
+	}
+	nfs_release_request(req);
+	return 0;
+}
+
 int __init nfs_init_readpagecache(void)
 {
 	nfs_rdata_cachep = kmem_cache_create("nfs_read_data",
diff -Nrup linux-2.6.29-rc5.org/include/linux/nfs_fs.h linux-2.6.29-rc5/include/linux/nfs_fs.h
--- linux-2.6.29-rc5.org/include/linux/nfs_fs.h	2009-02-16 12:31:18.000000000 +0900
+++ linux-2.6.29-rc5/include/linux/nfs_fs.h	2009-02-16 13:05:29.000000000 +0900
@@ -506,6 +506,9 @@ extern int  nfs_readpages(struct file *,
 		struct list_head *, unsigned);
 extern int  nfs_readpage_result(struct rpc_task *, struct nfs_read_data *);
 extern void nfs_readdata_release(void *data);
+extern int  nfs_is_partially_uptodate(struct page *, read_descriptor_t *,
+		unsigned long);
+
 
 /*
  * Allocate nfs_read_data structures

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/