linux-kernel - Re: [PATCH RFC V8 0/17] Paravirtualized ticket spinlocks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 13 May 2012 23:29:28 +0530
From:	Raghavendra K T <raghavendra.kt@...ux.vnet.ibm.com>
To:	Avi Kivity <avi@...hat.com>
CC:	Ingo Molnar <mingo@...nel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jeremy Fitzhardinge <jeremy@...p.org>,
	Greg Kroah-Hartman <gregkh@...e.de>,
	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
	"H. Peter Anvin" <hpa@...or.com>,
	Marcelo Tosatti <mtosatti@...hat.com>, X86 <x86@...nel.org>,
	Gleb Natapov <gleb@...hat.com>, Ingo Molnar <mingo@...hat.com>,
	Attilio Rao <attilio.rao@...rix.com>,
	Srivatsa Vaddagiri <vatsa@...ux.vnet.ibm.com>,
	Virtualization <virtualization@...ts.linux-foundation.org>,
	Xen Devel <xen-devel@...ts.xensource.com>,
	linux-doc@...r.kernel.org, KVM <kvm@...r.kernel.org>,
	Andi Kleen <andi@...stfloor.org>,
	Stefano Stabellini <stefano.stabellini@...citrix.com>,
	Stephan Diestelhorst <stephan.diestelhorst@....com>,
	LKML <linux-kernel@...r.kernel.org>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH RFC V8 0/17] Paravirtualized ticket spinlocks

On 05/07/2012 05:36 PM, Avi Kivity wrote:
> On 05/07/2012 01:58 PM, Raghavendra K T wrote:
>> On 05/07/2012 02:02 PM, Avi Kivity wrote:
>>> On 05/07/2012 11:29 AM, Ingo Molnar wrote:
>> (Less is better. Below is time elapsed in sec for x86_64_defconfig
>> (3+3 runs)).
>>
>>           BASE                    BASE+patch            %improvement
>>           mean (sd)               mean (sd)
>> case 1x:     66.0566 (74.0304)      61.3233 (68.8299)     7.16552
>> case 2x:     1253.2 (1795.74)      131.606 (137.358)     89.4984
>> case 3x:     3431.04 (5297.26)      134.964 (149.861)     96.0664
>>
>
> You're calculating the improvement incorrectly.  In the last case, it's
> not 96%, rather it's 2400% (25x).  Similarly the second case is about
> 900% faster.
>

speedup calculation is clear.

I think confusion for me was more because of the types of benchmarks.

I always did

|(patch - base)| * 100 / base


So,  for
(1) lesser is better sort of benchmarks,
improvement calculation would be like

|(patched -  base)| * 100/ patched
e.g for kernbench,

suppose base    = 150 sec
         patched = 100 sec
improvement = 50 % ( = 33% degradation of base)


(2) for higher is better sort of benchmarks improvement calculation 
would be like

|(patched - base)| * 100 / base

for e.g say for pgbench/ ebizzy...

     base = 100 tps (transactions per sec)
     patched = 150 tps

  improvement  = 50 % of pathched kernel ( OR 33 % degradation of base )


Is this is what generally done? just wanted to be on same page before 
publishing benchmark results, other than kernbench.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/