| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <878trk8urx.fsf@yhuang-dev.intel.com>
Date: Tue, 13 Dec 2016 09:47:30 +0800
From: "Huang\, Ying" <ying.huang@...el.com>
To: Vincent Guittot <vincent.guittot@...aro.org>
Cc: Stephen Rothwell <sfr@...b.auug.org.au>,
Andi Kleen <ak@...ux.intel.com>,
Tim Chen <tim.c.chen@...ux.intel.com>,
Peter Zijlstra <peterz@...radead.org>, LKP <lkp@...org>,
LKML <linux-kernel@...r.kernel.org>,
Dietmar Eggemann <dietmar.eggemann@....com>,
Dave Hansen <dave.hansen@...el.com>,
"Thomas Gleixner" <tglx@...utronix.de>,
Linus Torvalds <torvalds@...ux-foundation.org>,
Ingo Molnar <mingo@...nel.org>
Subject: Re: [LKP] [lkp-developer] [sched/fair] 4e5160766f: +149% ftq.noise.50% regression
Hi, Vincent,
Vincent Guittot <vincent.guittot@...aro.org> writes:
> Hi Ying,
>
> On 12 December 2016 at 06:43, kernel test robot
> <ying.huang@...ux.intel.com> wrote:
>> Greeting,
>>
>> FYI, we noticed a 149% regression of ftq.noise.50% due to commit:
>>
>>
>> commit: 4e5160766fcc9f41bbd38bac11f92dce993644aa ("sched/fair: Propagate asynchrous detach")
>> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
>>
>> in testcase: ftq
>> on test machine: 8 threads Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory
>> with following parameters:
>>
>> nr_task: 100%
>> samples: 6000ss
>> test: cache
>> freq: 20
>> cpufreq_governor: powersave
>
> Why using powersave ? Are you testing every governors ?
We will test performance and powersave governor for FTQ.
>>
>> test-description: The FTQ benchmarks measure hardware and software interference or 'noise' on a node from the applications perspective.
>> test-url: https://github.com/rminnich/ftq
>
> It's a bit difficult to understand exactly what is measured and what
> is ftq.noise.50% because this result is not part of the bench which
> seems to only record a log of data in a file and ftq.noise.50% seems
> to be lkp specific
Yes. FTQ itself has no noise statistics builtin, although it is an OS
noise benchmark. ftq.noise.50% is calculated as below:
There is a score for every sample of ftq. The lower the score, the
higher the noises. ftq.noise.50% is the number (per 1000000 samples) of
samples whose score is less than 50% of the mean score.
Best Regards,
Huang, Ying
> I have tried to reproduce the lkp test on a debian jessie then a
> ubuntu server 16.10 but lkp doesn't seems to install cleanly as there
> are some errors:
>
> sudo bin/lkp run job.yaml
> IPMI BMC is not supported on this machine, skip bmc-watchdog setup!
> 2016-12-12 13:58:39 ./ftq_cache -f 20 -n 6000 -t 8 -a 524288
> Start 5088418680237 end 5438443372098 elapsed 350024691861
> cyclestart 14236344834332 cycleend 15214154208877 elapsed 977809374545
> Avg Cycles(ticks) per ns. is 2.793544; nspercycle is 0.357968
> Pre-computed ticks per ns: 2.793541
> Sample frequency is 20.000000
> ticks per ns 2.79354
> chown: utilisateur incorrect: «lkp.lkp»
> chown: utilisateur incorrect: «lkp.lkp»
> wait for background monitors: 9405 9407 oom-killer nfs-hang
> curl: (6) Could not resolve host: ftq.time
>
>
>>
>> In addition to that, the commit also has significant impact on the following tests:
>>
>> +------------------+--------------------------------------------------------------------------------+
>> | testcase: change | unixbench: unixbench.score 2.7% improvement |
>> | test machine | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory |
>> | test parameters | cpufreq_governor=performance |
>> | | nr_task=100% |
>> | | runtime=300s |
>> | | test=execl |
>> +------------------+--------------------------------------------------------------------------------+
>>
>>
>> Details are as below:
>> -------------------------------------------------------------------------------------------------->
>>
>>
>> To reproduce:
>>
>> git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
>> cd lkp-tests
>> bin/lkp install job.yaml # job file is attached in this email
>> bin/lkp run job.yaml
>>
>> testcase/path_params/tbox_group/run: ftq/100%-6000ss-cache-20-powersave/lkp-hsw-d01
>>
>> 09a43ace1f986b00 4e5160766fcc9f41bbd38bac11
>> ---------------- --------------------------
>> %stddev change %stddev
>> \ | \
>> 305 ± 30% 260% 1100 ± 14% ftq.noise.75%
>> 1386 ± 19% 149% 3457 ± 7% ftq.noise.50%
>> 2148 ± 11% 98% 4257 ± 4% ftq.noise.25%
>> 3963589 3898578 ftq.time.involuntary_context_switches
>>
>>
>>
>> ftq.noise.50_
>>
>> 4000 ++------------O------------------------------------------------------+
>> | O O |
>> 3500 ++ O O O O O O O
>> | O O O O O O O O O O O O O O |
>> O O O O O |
>> 3000 ++ O |
>> | O |
>> 2500 ++ |
>> | |
>> 2000 ++ |
>> | * .* |
>> | + : * * * + |
>> 1500 ++ + : + + + + : + .* |
>> |.* *. + * *.. : * + |
>> 1000 *+-------*-----------*----------*------------------------------------+
>>
>> [*] bisect-good sample
>> [O] bisect-bad sample
>>
>>
>> Disclaimer:
>> Results have been estimated based on internal Intel analysis and are provided
>> for informational purposes only. Any difference in system hardware or software
>> design or configuration may affect actual performance.
>>
>>
>> Thanks,
>> Ying Huang
> _______________________________________________
> LKP mailing list
> LKP@...ts.01.org
> https://lists.01.org/mailman/listinfo/lkp
Powered by blists - more mailing lists