linux-kernel - Re: [PATCH] expand micro-optimizations in kernel to newer model CPUs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 15 Dec 2013 18:31:51 +0100
From:	Richard Weinberger <richard@....at>
To:	John <da_audiophile@...oo.com>
Cc:	Richard Weinberger <richard.weinberger@...il.com>,
	"H. Peter Anvin" <hpa@...or.com>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	"david.heidelberger@...t.cz" <david.heidelberger@...t.cz>,
	"tglx@...utronix.de" <tglx@...utronix.de>,
	"mingo@...hat.com" <mingo@...hat.com>,
	"x86@...nel.org" <x86@...nel.org>,
	lkml <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] expand micro-optimizations in kernel to newer model CPUs

Am Sonntag, 15. Dezember 2013, 04:42:50 schrieb John:
> ----- Original Message -----
> 
> > From: Richard Weinberger <>
> > 
> > A *very* small speedup.
> > 
> > And I really doubt your numbers.
> > Why are you using ANOVA? You're comparing *two* groups not more than two.
> > I had a quick look at your raw numbers, they don't seem to be normally
> > distributed at all.
> > Did you remove some peaks?
> 
> Hi Richard.  Thank you for your interest.  Yes, a small speedup as I
> mentioned but  I'll note that the current kernel code includes the MCORE2
> option.  I tested this against some of the newer ones and they are all on
> par with each other.  For example, here are differences in median values:
> 
> CPUDifference in median value
> core2        +87.5 ms
> core7-avx+79.7 ms
> core-avx-i+257.2 ms
> 
> I am using ANOVA to establish that the generic group differs from the
> optimized group.  I have always used ANOVA for this sort of comparison
> whether using two or more groups.  In fact, thumb through any medical or
> scientific journal, you'll see others in pier reviewed article doing the
> same.  

Only because others so does not make it valid.
Why not a plain T-test?

> I did not remove any datapoints; I do not understand why you don't think the
> sets are normally distributed.  Did you see the normal quantile plots?
>  Additionally, the population variances are fairly equal (Levene and
> Barlett tests). 

Just perform a simple Kolmogorow-Smirnow test like on 
http://jumk.de/statistic-calculator or http://www.physics.csbsju.edu/stats/KS-test.n.plot_form.html and you'll find out.
IIRC from my statistics 101 you'd have to perform a two-sided ANOVA test
if your data points are not normally distributed.

That said, we should not waste time with statistics games.
What we need are reliable and reproducible results.
As Boris requested many times before...

Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/