linux-kernel - [PATCH -tip 0/6] rwsem: Fine tuning

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1422257769-14083-1-git-send-email-dave@stgolabs.net>
Date:	Sun, 25 Jan 2015 23:36:03 -0800
From:	Davidlohr Bueso <dave@...olabs.net>
To:	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...nel.org>
Cc:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Jason Low <jason.low2@...com>,
	Michel Lespinasse <walken@...gle.com>,
	Tim Chen <tim.c.chen@...ux.intel.com>,
	linux-kernel@...r.kernel.org, Davidlohr Bueso <dave@...olabs.net>
Subject: [PATCH -tip 0/6] rwsem: Fine tuning

Hello,

First two patches are self descriptive obvious add-ons.

The rest are performance enhancements for write-mostly workloads.
While this is not our priority (use mutexes instead!!), there are
cases where heavy use of writers can severly hurt rwsem performance.
For instace, we got reports[1] of writer only issues when converting
the i_mmap_rwsem from mutex, on a workload that pathologically pounds
on this lock for vma operations:

-     81.20%           execl  [kernel.kallsyms]     [k] osq_lock
      - 100.00% mutex_optimistic_spin                                                                                                   
           __mutex_lock_slowpath                                                                                                        
         - mutex_lock                                                                                                                   
            + 47.71% unlink_file_vma                                                                                                    
            + 34.91% vma_adjust                                                                                                         
            + 17.38% vma_link

This is enough to make small differences painfully evident. These changes
(particularly patch 6/6) recover most (~75%) of the performance regression.
Patches 4 and 6 deal with optimistic spinning fine tunning, while patch
5 is an attempt to get tid of two barriers when blocking. While I believe
this is safe, it certainly needs more eyeballs, I could have easily overlooked
something. Most of these changes are straighforward, but have various implications.

Passes multiple x86-64 tests.

Thanks!

[1] https://lkml.org/lkml/2015/1/7/884

Davidlohr Bueso (6):
  locking/rwsem: Use task->state helpers
  locking/rwsem: Document barrier need when waking tasks
  locking/rwsem: Set lock ownership ASAP
  locking/rwsem: Avoid deceiving lock spinners
  locking/rwsem: Optimize slowpath/sleeping
  locking/rwsem: Check for active lock before bailing on spinning

 kernel/locking/mutex.c          |  2 +-
 kernel/locking/rwsem-spinlock.c |  7 +++-
 kernel/locking/rwsem-xadd.c     | 71 +++++++++++++++++++++++++++++------------
 kernel/locking/rwsem.c          | 22 +------------
 kernel/locking/rwsem.h          | 20 ++++++++++++
 5 files changed, 78 insertions(+), 44 deletions(-)
 create mode 100644 kernel/locking/rwsem.h

-- 
2.1.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/