linux-kernel - [LSF/MM TOPIC] Improve performance of fget/fput

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190215163852.6ls6bchssazma6bm@davejwatson-mba.local>
Date:   Fri, 15 Feb 2019 16:38:05 +0000
From:   Dave Watson <davejwatson@...com>
To:     "lsf-pc@...ts.linux-foundation.org" 
        <lsf-pc@...ts.linux-foundation.org>
CC:     "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        Maged Michael <magedmichael@...com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: [LSF/MM TOPIC] Improve performance of fget/fput

In some of our hottest network services, fget_light + fput overhead
can represent 1-2% of the processes' total CPU usage.  I'd like to
discuss ways to reduce this overhead.

One proposal we have been testing is removing the refcount increment
and decrement, and using some sort of safe memory reclamation
instead. The hottest callers include recvmsg, sendmsg, epoll_wait, etc
- mostly networking calls, often used on non-blocking sockets.  Often
we are only incrementing and decrementing the refcount for a very
short period of time, ideally we wouldn't adjust the refcount unless
we know we are going to block.

We could use RCU, but we would have to be particularly careful that
none of these calls ever block, or ensure that we increment the
refcount at the blocking locations.  As an alternative to RCU, hazard
pointers have similar overhead to SRCU, and could work equally well on
blocking or nonblocking syscalls without additional changes.

(There were also recent related discussions on SCM_RIGHTS refcount
cycle issues, which is the other half of a file* gc)

There might also be ways to rearrange the file* struct or fd table so
that we're not taking so many cache misses for sockfd_lookup_light,
since for sockets we don't use most of the file* struct at all.