linux-ext4 - Re: Kernel Benchmarking

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ed8442fd-6f54-dd84-cd4a-941e8b7ee603@MichaelLarabel.com>
Date:   Tue, 15 Sep 2020 09:21:58 -0500
From:   Michael Larabel <Michael@...haelLarabel.com>
To:     Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     Ext4 Developers List <linux-ext4@...r.kernel.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>
Subject: Re: Kernel Benchmarking

On 9/14/20 12:47 PM, Linus Torvalds wrote:
> Michael et al,
>   Ok, I redid my failed "hybrid mode" patch from scratch (original
> patch never sent out, I never got it to a working point).
>
> Having learnt from my mistake, this time instead of trying to mix the
> old and the new code, instead I just extended the new code, and wrote
> a _lot_ of comments about it.
>
> I also made it configurable, using a "page_lock_unfairness" knob,
> which this patch defaults to 1000 (which is basically infinite).
> That's just a value that says how many times we'll try the old unfair
> case, so "1000" means "we'll re-queue up to a thousand times before we
> say enough is enough" and zero is the fair mode that shows the
> performance problems.
>
> I've only (lightly) tested those two extremes, I think the interesting
> range is likely in the 1-5 range.
>
> So you can do
>
>      echo 0 > /proc/sys/vm/page_lock_unfairness
>      .. run test ..
>
> and you should get the same numbers as without this patch (within
> noise, of course).
>
> Or do
>
>      echo 5 > /proc/sys/vm/page_lock_unfairness
>      .. run test ..
>
> and get numbers for "we accept some unfairness, but if we have to
> requeue more than five times, we force the fair mode".
>
> Again, the default is 1000, which is ludicrously high (it's not a
> "this many retries per page" count, it's a "for each waiter" count). I
> made it that high just because I have *not* run any numbers for that
> interesting range, I just checked the extreme cases, and I wanted to
> make sure that Michael sees the old performance (modulo other changes
> to 5.9, of course).
>
> Comments? The patch really has a fair amount of comments in it, in
> fact the code changes are reasonably small, most of the changes really
> are about new and updated comments about what is going on.
>
> I was burnt by making a mess of this the first time, so I proceeded
> more thoughtfully this time. Hopefullyt the end result is also better.
>
> (Note that it's a commit and has a SHA1, but it's from my "throw-away
> tree for testing", so it doesn't have my sign-off or any real commit
> message yet: I'll do that once it gets actual testing and comments).
>
>                   Linus


Still running more benchmarks and on more systems, but so far at least 
as the Apache test is concerned this patch does seem to largely address 
the issue. The performance with the default 1000 page_lock_unfairness 
was yielding results more similar to 5.8 and in some cases tweaking the 
value did help improve the performance. A PLU value of 4~5 seems to 
yield the best performance.

The results though with Hackbench and Redis that exhibited similar drops 
from the commit in question remained mixed. Overview of the Apache 
metrics below, the other tests and system details @ 
https://openbenchmarking.org/result/2009154-FI-LINUX58CO57 For the other 
systems still testing, so far it's looking like similar relative impact 
to these results.

Apache Siege 2.4.29
Concurrent Users: 1
Transactions Per Second > Higher Is Better
v5.8 ............. 7684.81 
|=================================================
v5.9 Git ......... 7390.86 |===============================================
Default PLU 1000 . 7579.49 
|=================================================
PLU 0 ............ 7937.84 
|===================================================
PLU 1 ............ 7464.61 |================================================
PLU 2 ............ 7552.61 
|=================================================
PLU 3 ............ 7475.96 |================================================
PLU 4 ............ 7638.69 
|=================================================
PLU 5 ............ 7735.75 
|==================================================


Apache Siege 2.4.29
Concurrent Users: 50
Transactions Per Second > Higher Is Better
v5.8 ............. 39280.51 |===============================================
v5.9 Git ......... 28240.71 |==================================
Default PLU 1000 . 39708.30 |===============================================
PLU 0 ............ 26645.15 |================================
PLU 1 ............ 38709.95 |==============================================
PLU 2 ............ 39712.82 |===============================================
PLU 3 ............ 41959.67 
|==================================================
PLU 4 ............ 38870.90 |==============================================
PLU 5 ............ 41301.97 
|=================================================


Apache Siege 2.4.29
Concurrent Users: 100
Transactions Per Second > Higher Is Better
v5.8 ............. 51255.73 |=============================================
v5.9 Git ......... 21926.62 |===================
Default PLU 1000 . 42001.86 |=====================================
PLU 0 ............ 21528.43 |===================
PLU 1 ............ 37138.49 |=================================
PLU 2 ............ 38086.58 |==================================
PLU 3 ............ 38057.72 |==================================
PLU 4 ............ 56350.51 
|==================================================
PLU 5 ............ 37868.57 |==================================


Apache Siege 2.4.29
Concurrent Users: 200
Transactions Per Second > Higher Is Better
v5.8 ............. 47825.12 |===================================
v5.9 Git ......... 20174.78 |===============
Default PLU 1000 . 48190.05 |===================================
PLU 0 ............ 20095.10 |===============
PLU 1 ............ 48524.44 |====================================
PLU 2 ............ 47823.09 |===================================
PLU 3 ............ 47751.02 |===================================
PLU 4 ............ 68286.02 
|==================================================
PLU 5 ............ 47662.08 |===================================


Apache Siege 2.4.29
Concurrent Users: 250
Transactions Per Second > Higher Is Better
v5.8 ............. 55279.65 |=====================================
v5.9 Git ......... 20282.62 |==============
Default PLU 1000 . 67639.46 |=============================================
PLU 0 ............ 20181.98 |==============
PLU 1 ............ 40505.37 |===========================
PLU 2 ............ 56914.07 |======================================
PLU 3 ............ 55285.35 |=====================================
PLU 4 ............ 55499.25 |=====================================
PLU 5 ............ 74347.77 
|==================================================