linux-kernel - Re: collecting simple benchmark scripts?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 15 Feb 2018 10:32:21 -0800
From:   Laura Abbott <labbott@...hat.com>
To:     Kees Cook <keescook@...gle.com>, Shuah Khan <shuah@...nel.org>,
        Fengguang Wu <fengguang.wu@...el.com>,
        Tycho Andersen <tycho@...ho.ws>,
        Steven Rostedt <srostedt@...hat.com>,
        Kevin Hilman <khilman@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
Cc:     LKML <linux-kernel@...r.kernel.org>
Subject: Re: collecting simple benchmark scripts?

On 02/14/2018 02:50 PM, Kees Cook wrote:
> Hi,
> 
> In a separate thread, some folks were looking for some simple
> benchmarks for evaluating various changes to kernel internals (as
> opposed to the much more focused things like xfstests). For me, this
> has been an area of lore and passed-around scripts, and it seems like
> maybe we should have a subdirectory of tools/testing/benchmarks/ or
> something to collect these?
> 
> (Or maybe this already exists and I've totally missed it?)
> 
> I've got at least one micro-benchmark in
> tools/testing/selftests/seccomp/seccomp_benchmark.c, and searches show
> tools/testing/selftests/vm/gup_benchmark.c too, but I was thinking of
> either more generali things more like the famous "kernel build
> benchmark" or a wrapper for running hackbench to get some statistics
> out of it, etc.
> 
> Or, I guess, at least collecting all the micro-benchmarks in some
> single place, as they're a bit scattered.
> 
> I'm sure I'm not remotely the first person to bring this up, but my
> attempts at searches for this have failed.
> 
> Thoughts?
> 
> -Kees
> 

This is the script I've been using for hackbench since it can
be noisy. I expect someone to tell me the math is wrong but
I'd be happy to throw this in a repo if others are interested

#!/bin/sh

CNT=100
mean=0.0
M2=0.0
for i in $(seq 1 $CNT); do
	echo $i
	r=`hackbench -g 20 -l 1000 | grep Time | cut -d ' ' -f 2`
	d_calc="$r-$mean"
         d=`echo $d_calc | bc -l`
	mean_calc="$mean+($d/$i)"
	mean=`echo $mean_calc | bc -l`
	M2_calc="$M2+($d*($r-$mean))"
	M2=`echo  $M2_calc| bc -l`
done

echo "mean $mean"
V_calc="$M2/$(($CNT-1))"
V=`echo $V_calc | bc -l`
DEV_calc="sqrt($V)"
DEV=`echo $DEV_calc | bc -l`
echo "variance $V"
echo "stdev $DEV"