linux-kernel - [RFC][PATCH 7/9] vfs: Optimize fops_read

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <m1ws9r5r00.fsf@fess.ebiederm.org>
Date:	Sat, 11 Apr 2009 05:11:59 -0700
From:	ebiederm@...ssion.com (Eric W. Biederman)
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	<linux-kernel@...r.kernel.org>, <linux-pci@...r.kernel.org>,
	<linux-mm@...ck.org>, <linux-fsdevel@...r.kernel.org>,
	Al Viro <viro@...IV.linux.org.uk>,
	Hugh Dickins <hugh@...itas.com>, Tejun Heo <tj@...nel.org>,
	Alexey Dobriyan <adobriyan@...il.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Alan Cox <alan@...rguk.ukuu.org.uk>,
	Greg Kroah-Hartman <gregkh@...e.de>
Subject: [RFC][PATCH 7/9] vfs: Optimize fops_read_lock


After seeing fget_light and the justification for it in the commit
log I did not want to introduce something into the common read/write
path of file descriptors that could have a significant measurable impact
on I/O speed.

commit f6435db01709533f270b2dce1e5914770dbc65de
Author: akpm <akpm>
Date:   Thu May 8 05:19:50 2003 +0000

    [PATCH] reduced overheads in fget/fput

    From: Dipankar Sarma <dipankar@...ibm.com>

    fget() shows up on profiles, especially on SMP.  Dipankar's patch
    special-cases the situation wherein there are no sharers of current->files.

    In this situation we know that no other process can close this file, so it
    is not necessary to increment the file's refcount.

    It's ugly as sin, but makes a substantial difference.

    The test is

        dd if=/dev/zero of=foo bs=1 count=1M

    On 4CPU P3 xeon with 1MB L2 cache and 512MB ram:

        kernel           sys time     std-dev
        ------------     --------     -------

        UP - vanilla     2.104        0.028
        UP - file        1.867        0.019

        SMP - vanilla    2.976        0.023
        SMP - file       2.719        0.026

    BKrev: 3eb9e8f6Db0nMWoSx5IdHx6SBal8aw

My test case was:
    dd if=/dev/zero of=/dev/null bs=1 count=1M.

As writing to a real file turned out to cover the cost.

Without the optimization I am seeing 2.4 - 2.5 MB/s on my
idle core2 single socket quad core.

With this optimization I am seeing 2.9 - 3.0 MB/s on the
same machine.  Maybe 2% slower than before I introduced
fops_read_lock.

The common case is that there is only one thread and so
the fget_light optimization applies and f_count remains
at 1.  Which implies that there is only a single process
performing operations through the file descriptor.

In that case because there is no possible contention
it is possible safely skip the atomic operations, gaining all
of the benefits of rcu without requiring a per cpu variable.

Signed-off-by: Eric W. Biederman <ebiederm@...ssion.com>
---
 fs/file_table.c |   18 ++++++++++++++----
 1 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/fs/file_table.c b/fs/file_table.c
index d216557..634d44c 100644
--- a/fs/file_table.c
+++ b/fs/file_table.c
@@ -495,15 +495,25 @@ void file_kill(struct file *file)
 int fops_read_lock(struct file *file)
 {
 	int revoked = (file->f_mode & FMODE_REVOKED);
-	if (likely(!revoked))
-		atomic_long_inc(&file->f_use);
+	if (likely(!revoked)) {
+		if (likely(atomic_long_read(&file->f_count) == 1))
+			atomic_long_set(&file->f_use,
+				atomic_long_read(&file->f_use) + 1);
+		else
+			atomic_long_inc(&file->f_use);
+	}
 	return revoked;
 }
 
 void fops_read_unlock(struct file *file, int revoked)
 {
-	if (likely(!revoked))
-		atomic_long_dec(&file->f_use);
+	if (likely(!revoked)) {
+		if (likely(atomic_long_read(&file->f_count) == 1))
+			atomic_long_set(&file->f_use,
+				atomic_long_read(&file->f_use) - 1);
+		else
+			atomic_long_dec(&file->f_use);
+	}
 }
 
 int fs_may_remount_ro(struct super_block *sb)
-- 
1.6.1.2.350.g88cc

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/