linux-kernel - [PATCH v2 0/4] use rwlock in order to reduce ep_poll

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Thu,  3 Jan 2019 16:01:00 +0100
From:   Roman Penyaev <rpenyaev@...e.de>
To:     unlisted-recipients:; (no To-header on input)
Cc:     Roman Penyaev <rpenyaev@...e.de>, Davidlohr Bueso <dbueso@...e.de>,
        Jason Baron <jbaron@...mai.com>,
        Al Viro <viro@...iv.linux.org.uk>,
        "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [PATCH v2 0/4] use rwlock in order to reduce ep_poll_callback() contention

The last patch targets the contention problem in ep_poll_callback(), which
can be very well reproduced by generating events (write to pipe or eventfd)
from many threads, while consumer thread does polling.

The following are some microbenchmark results based on the test [1] which
starts threads which generate N events each.  The test ends when all events
are successfully fetched by the poller thread:

 spinlock
 ========

 threads  events/ms  run-time ms
       8       6402        12495
      16       7045        22709
      32       7395        43268

 rwlock + xchg
 =============

 threads  events/ms  run-time ms
       8      10038         7969
      16      12178        13138
      32      13223        24199

According to the results bandwidth of delivered events is significantly
increased, thus execution time is reduced.

This series is based on linux-next/akpm.

v2:
 o I was wrong saying that ep_poll_callback() can't be called 
   concurrently for the same epi: several wait queues can be
   attached to the single epoll item, thus several event sources
   can signal in parallel.  To cover this case lockless element
   addition has to detect that the same @epi is not yet in the
   list.  This is done by extra cmpxchg() operation.

 o unify awaking of wakeup source calling ep_pm_stay_awake_rcu(epi)
   in all the cases from ep_poll_callback() path.

 o more explicit comments

[1] https://github.com/rouming/test-tools/blob/master/stress-epoll.c

Roman Penyaev (4):
  epoll: make sure all elements in ready list are in FIFO order
  epoll: loosen irq safety in ep_poll_callback()
  epoll: unify awaking of wakeup source on ep_poll_callback() path
  epoll: use rwlock in order to reduce ep_poll_callback() contention

 fs/eventpoll.c | 178 ++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 133 insertions(+), 45 deletions(-)

Signed-off-by: Roman Penyaev <rpenyaev@...e.de>
Cc: Davidlohr Bueso <dbueso@...e.de>
Cc: Jason Baron <jbaron@...mai.com>
Cc: Al Viro <viro@...iv.linux.org.uk>
Cc: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: linux-fsdevel@...r.kernel.org
Cc: linux-kernel@...r.kernel.org
-- 
2.19.1