[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <YeAFyOR61+c+FyMn@kroah.com>
Date: Thu, 13 Jan 2022 11:58:16 +0100
From: Greg KH <gregkh@...uxfoundation.org>
To: Imran Khan <imran.f.khan@...cle.com>
Cc: tj@...nel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 2/2] kernfs: Reduce contention around global per-fs
kernfs_rwsem.
On Thu, Jan 13, 2022 at 09:42:59PM +1100, Imran Khan wrote:
> Right now a global per file system based rwsem (kernfs_rwsem)
> synchronizes multiple kernfs operations. On a large system with
> few hundred CPUs and few hundred applications simultaenously trying
> to access sysfs, this results in multiple sys_open(s) contending on
> kernfs_rwsem via kernfs_iop_permission and kernfs_dop_revalidate.
>
> - 21.42% 21.34% showgids [kernel.kallsyms] [k] up_read
> 21.34% __libc_start_main
> __GI___libc_open
> entry_SYSCALL_64_after_hwframe
> do_syscall_64
> sys_open
> do_sys_open
> do_filp_open
> - path_openat
> - 20.05% link_path_walk
> - 9.76% walk_component
> lookup_fast
> - d_revalidate.part.24
> - 9.75% kernfs_dop_revalidate
> up_read
> - 9.46% inode_permission
> - __inode_permission
> - 9.46% kernfs_iop_permission
> up_read
> - 0.83% kernfs_iop_get_link
> up_read
> - 0.80% lookup_fast
> d_revalidate.part.24
> kernfs_dop_revalidate
> up_read
>
> - 21.31% 21.21% showgids [kernel.kallsyms] [k] down_read
> 21.21% __libc_start_main
> __GI___libc_open
> entry_SYSCALL_64_after_hwframe
> do_syscall_64
> sys_open
> do_sys_open
> do_filp_open
> - path_openat
> - 19.78% link_path_walk
> - 10.62% inode_permission
> - __inode_permission
> - 10.62% kernfs_iop_permission
> down_read
> - 8.45% walk_component
> lookup_fast
> - d_revalidate.part.24
> - 8.45% kernfs_dop_revalidate
> down_read
> - 0.71% kernfs_iop_get_link
> down_read
> - 0.72% lookup_fast
> - d_revalidate.part.24
> - 0.72% kernfs_dop_revalidate
> down_read
> - 0.71% may_open
> inode_permission
> __inode_permission
> kernfs_iop_permission
> down_read
>
> Since permission is specific to a kernfs_node we can use a hashed
> lock to access/modify permission. Also use kernfs reference counting
> to ensure we are accessing/modifying permissions for an existing
> kernfs_node object.
>
> Using this change brings down the above mentioned down_read/up_read
> numbers to ~8%, thus indicating that contention around kernfs_rwsem
> has reduced to about 1/3rd of earlier value.
Ah, nevermind, you do post the results here, I should have kept reading.
Nice work!
I'll look at these after 5.17-rc1 is out, thanks!
greg k-h
Powered by blists - more mailing lists