linux-kernel - NFS: low read/stat performance on small files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAKX=r8A90b25VKXC1qtAota9DqDrj37xFT511DuvrZ=1Eux6Xw@mail.gmail.com>
Date:	Fri, 23 Mar 2012 16:47:04 +0530
From:	Vivek Trivedi <vtrivedi018@...il.com>
To:	"Myklebust, Trond" <Trond.Myklebust@...app.com>,
	linux-nfs@...r.kernel.org, linux-kernel@...r.kernel.org,
	Namjae Jeon <linkinjeon@...il.com>
Cc:	vtrivedi018@...il.com, amit.sahrawat83@...il.com
Subject: NFS: low read/stat performance on small files

Hi,
we are facing below 2 performance issue on NFS:

1. Read speed is low for small files
==========================

[ Log on NFS Client]

$ echo 3 > /proc/sys/vm/drop_caches
$ dd if=200KBfile.txt of=/dev/null
400+0 records in
400+0 records out
204800 bytes (200.0KB) copied, 0.027074 seconds, 7.2MB/s

Read speed for 200KB file is 7.2 MB

[ Log on NFS Client]

$ echo 3 > /proc/sys/vm/drop_caches
$ dd if=100MBfile.txt of=/dev/null
204800+0 records in
204800+0 records out
104857600 bytes (100.0MB) copied, 9.351221 seconds, 10.7MB/s

Read speed for 100MB file is 10.7 MB

As you see read speed for 200KB file is only 7.2MB/sec while it is
10.7 MB/sec when we read 100MB file.
Why there is so much difference in read performance ?
Is there any way to achieve high read speed for small files ?

2. Read/stat for a directory tree is slow on NFS than local
==========================================

we have lot of *.jpg files in a directory. If we try to "stat" and
"read" from this directory,
 performannce is very slow on NFS Client compared to Local(NFS server)
"stat" and "read"

[ Log on Local (NFS Server) ]

$ echo 3 > /proc/sys/vm/drop_caches
$ ./stat_read_files_test ./lot_of_jpg_files/
 Time Taken : 9288 msec

[ Log on NFS Client]

$ echo 3 > /proc/sys/vm/drop_caches
$ ./stat_read_files_test ./lot_of_jpg_files/
 Time Taken : 19966 msec

As you see, on NFS client time taken is almost *double* than that of
local(NFS server)
We are using UDP with rsize,wsize=32k on 100MB ethernet link.
I am attaching read/stat testcase.

Is there any way to improve this performance ?

Thanks,
Vivek

View attachment "traversepath.c" of type "text/x-csrc" (2188 bytes)