| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 2 Jun 2021 15:29:26 +0800
From: kernel test robot <oliver.sang@...el.com>
To: Roman Gushchin <guro@...com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>,
Andrew Morton <akpm@...ux-foundation.org>,
Vlastimil Babka <vbabka@...e.cz>,
Shakeel Butt <shakeelb@...gle.com>,
Christoph Lameter <cl@...ux.com>,
Johannes Weiner <hannes@...xchg.org>,
Michal Hocko <mhocko@...nel.org>, Tejun Heo <tj@...nel.org>,
LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
zhengjun.xing@...ux.intel.com
Subject: [mm] 286e04b8ed: will-it-scale.per_thread_ops -8.0% regression
Greeting,
FYI, we noticed a -8.0% regression of will-it-scale.per_thread_ops due to commit:
commit: 286e04b8ed7a04279ae277f0f024430246ea5eec ("mm: memcg/slab: allocate obj_cgroups for non-root slab pages")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
nr_task: 16
mode: thread
test: brk1
cpufreq_governor: performance
ucode: 0x5003006
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
In addition to that, the commit also has significant impact on the following tests:
+------------------+-------------------------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -19.7% regression |
| test machine | 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=socket |
| | iterations=4 |
| | mode=threads |
| | nr_threads=100% |
| | ucode=0x5003006 |
+------------------+-------------------------------------------------------------------------------------+
| testcase: change | hackbench: boot-time.boot 10.6% regression |
| test machine | 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=socket |
| | iterations=4 |
| | mode=process |
| | nr_threads=100% |
| | ucode=0x5003003 |
+------------------+-------------------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
bin/lkp run generated-yaml-file
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/thread/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/brk1/will-it-scale/0x5003006
commit:
bf4f059954 ("mm: memcg/slab: obj_cgroup API")
286e04b8ed ("mm: memcg/slab: allocate obj_cgroups for non-root slab pages")
bf4f059954dcb221 286e04b8ed7a04279ae277f0f02
---------------- ---------------------------
%stddev %change %stddev
\ | \
1259202 -8.0% 1159095 ± 5% will-it-scale.16.threads
78699 -8.0% 72443 ± 5% will-it-scale.per_thread_ops
1259202 -8.0% 1159095 ± 5% will-it-scale.workload
0.11 ± 5% -0.0 0.10 ± 7% mpstat.cpu.all.soft%
168284 ± 14% -37.2% 105678 ± 43% numa-vmstat.node3.numa_other
4010690 +8.6% 4353788 ± 5% perf-stat.overall.path-length
10386 +1.4% 10527 proc-vmstat.nr_mapped
2118886 ± 7% -16.2% 1775301 ± 8% sched_debug.cfs_rq:/.min_vruntime.max
1879470 ± 9% -20.0% 1504476 ± 10% sched_debug.cfs_rq:/.spread0.max
4.09 ± 4% +9.6% 4.48 ± 2% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
292.83 ± 4% -67.9% 94.00 ±141% perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
2449 ± 3% -9.0% 2230 ± 2% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
4.07 ± 3% +9.7% 4.46 ± 2% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
41872 ± 4% -14.9% 35652 ± 16% softirqs.CPU30.SCHED
41359 ± 2% -12.7% 36103 ± 17% softirqs.CPU49.SCHED
42091 ± 3% -6.8% 39230 ± 3% softirqs.CPU62.SCHED
42051 ± 3% -6.9% 39166 ± 3% softirqs.CPU67.SCHED
0.01 ±223% +0.0 0.06 ± 13% perf-profile.children.cycles-pp.run_local_timers
0.21 ± 15% +0.1 0.33 ± 20% perf-profile.children.cycles-pp.anon_vma_clone
1.00 ± 9% -0.2 0.75 ± 13% perf-profile.self.cycles-pp.rwsem_optimistic_spin
0.07 ± 20% +0.1 0.13 ± 21% perf-profile.self.cycles-pp.anon_vma_clone
0.24 ± 9% +0.1 0.31 ± 12% perf-profile.self.cycles-pp.__vma_adjust
130.17 ± 22% +75.2% 228.00 ± 22% interrupts.CPU0.RES:Rescheduling_interrupts
5270 ± 21% -32.3% 3570 ± 30% interrupts.CPU109.NMI:Non-maskable_interrupts
5270 ± 21% -32.3% 3570 ± 30% interrupts.CPU109.PMI:Performance_monitoring_interrupts
117.83 ± 17% +19.2% 140.50 ± 7% interrupts.CPU134.NMI:Non-maskable_interrupts
117.83 ± 17% +19.2% 140.50 ± 7% interrupts.CPU134.PMI:Performance_monitoring_interrupts
861.00 +56.9% 1351 ± 42% interrupts.CPU188.CAL:Function_call_interrupts
66.67 ± 42% +59.2% 106.17 ± 27% interrupts.CPU23.NMI:Non-maskable_interrupts
66.67 ± 42% +59.2% 106.17 ± 27% interrupts.CPU23.PMI:Performance_monitoring_interrupts
260.00 ± 26% +45.8% 379.00 ± 21% interrupts.CPU95.NMI:Non-maskable_interrupts
260.00 ± 26% +45.8% 379.00 ± 21% interrupts.CPU95.PMI:Performance_monitoring_interrupts
23771 ± 12% -25.1% 17793 ± 19% interrupts.CPU98.CAL:Function_call_interrupts
will-it-scale.per_thread_ops
85000 +-------------------------------------------------------------------+
| .+ |
| + + + .+ ++ +. |
80000 |:.+ + +.+ :+ +.+++ :+ :+ + + :+ +.+ .+ : + +.+|
|+ :+: : ++.+ + + : : + + + : + + : + +.: :.+ |
| + : + : : : : + O + + + |
75000 |-+ :: :: : : O O |
| O::O + :: OO OO O |
70000 |-+OO + O O :: OO |
| O O OO :O |
| O O O O O + |
65000 |O+ |
| O |
| O |
60000 +-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2ap4: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-9/performance/socket/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap4/hackbench/0x5003006
commit:
bf4f059954 ("mm: memcg/slab: obj_cgroup API")
286e04b8ed ("mm: memcg/slab: allocate obj_cgroups for non-root slab pages")
bf4f059954dcb221 286e04b8ed7a04279ae277f0f02
---------------- ---------------------------
%stddev %change %stddev
\ | \
200843 ± 5% -19.7% 161359 ± 4% hackbench.throughput
4.774e+09 ± 4% +6.5% 5.084e+09 ± 2% hackbench.time.involuntary_context_switches
4361 ± 7% +9.4% 4771 ± 5% hackbench.time.user_time
6.849e+09 ± 2% +6.2% 7.273e+09 ± 2% hackbench.time.voluntary_context_switches
0.00 ±115% +0.0 0.00 ± 50% mpstat.cpu.all.iowait%
0.03 ± 2% +0.0 0.04 ± 15% mpstat.cpu.all.soft%
55921 ± 31% +44.1% 80594 ± 10% numa-numastat.node2.other_node
86786 -52.4% 41350 ± 77% numa-numastat.node3.other_node
7930 ± 12% +28.7% 10208 ± 6% slabinfo.eventpoll_pwq.active_objs
7930 ± 12% +28.7% 10208 ± 6% slabinfo.eventpoll_pwq.num_objs
1.726e+08 ± 25% +38.1% 2.384e+08 ± 12% cpuidle.C1.time
11923928 ± 26% +40.4% 16738105 ± 10% cpuidle.C1.usage
8e+08 ± 35% +36.1% 1.089e+09 ± 3% cpuidle.C1E.time
3885779 ± 16% +30.1% 5054945 ± 8% cpuidle.C1E.usage
2995540 ± 27% +51.7% 4542950 ± 9% cpuidle.POLL.time
1099556 ± 26% +53.2% 1685053 ± 9% cpuidle.POLL.usage
5087256 ± 23% +74.8% 8894478 ± 25% turbostat.C1
0.06 ± 22% +0.1 0.11 ± 30% turbostat.C1%
1645987 ± 9% +69.4% 2787738 ± 26% turbostat.C1E
0.33 ± 32% +0.1 0.47 ± 9% turbostat.C1E%
7.906e+08 ± 2% +16.4% 9.202e+08 ± 6% turbostat.IRQ
61.67 ± 3% +4.3% 64.33 turbostat.PkgTmp
329726 ± 32% +113.5% 703843 ± 43% numa-meminfo.node0.FilePages
19643 ±108% +412.4% 100661 ± 60% numa-meminfo.node0.Inactive
19643 ±108% +412.4% 100661 ± 60% numa-meminfo.node0.Inactive(anon)
89287 ±130% +399.6% 446057 ± 70% numa-meminfo.node0.Shmem
42820 ± 18% -37.4% 26822 ± 16% numa-meminfo.node2.KReclaimable
42820 ± 18% -37.4% 26822 ± 16% numa-meminfo.node2.SReclaimable
153505 ± 43% -43.0% 87443 ± 7% numa-meminfo.node2.SUnreclaim
196325 ± 37% -41.8% 114266 ± 2% numa-meminfo.node2.Slab
80741 ± 26% +241.4% 275684 ± 47% numa-meminfo.node3.AnonPages.max
799.33 ± 28% +120.8% 1764 ± 40% numa-meminfo.node3.PageTables
4.072e+08 ± 2% -3.2% 3.942e+08 ± 3% perf-stat.i.cache-misses
1569742 ± 5% +6.4% 1670136 ± 2% perf-stat.i.cpu-migrations
1481 ± 2% +3.2% 1528 ± 3% perf-stat.i.cycles-between-cache-misses
0.00 ± 7% -0.0 0.00 ± 3% perf-stat.i.dTLB-store-miss-rate%
1191539 ± 11% -24.8% 896157 ± 11% perf-stat.i.dTLB-store-misses
64065533 ± 4% -4.4% 61233574 ± 4% perf-stat.i.node-store-misses
1124963 ± 12% -22.1% 876883 ± 14% perf-stat.i.node-stores
0.00 ± 8% -0.0 0.00 ± 5% perf-stat.overall.dTLB-store-miss-rate%
1189270 ± 11% -25.3% 888021 ± 12% perf-stat.ps.dTLB-store-misses
63319469 ± 3% -4.9% 60243321 ± 4% perf-stat.ps.node-store-misses
1130269 ± 12% -21.8% 884361 ± 15% perf-stat.ps.node-stores
1.032e+14 ± 2% +5.3% 1.087e+14 ± 2% perf-stat.total.instructions
82471 ± 32% +113.1% 175770 ± 43% numa-vmstat.node0.nr_file_pages
5078 ±109% +397.9% 25286 ± 60% numa-vmstat.node0.nr_inactive_anon
22361 ±130% +397.8% 111323 ± 70% numa-vmstat.node0.nr_shmem
5078 ±109% +397.9% 25286 ± 60% numa-vmstat.node0.nr_zone_inactive_anon
1257447 ± 13% -22.8% 971273 ± 11% numa-vmstat.node1.numa_local
84779 ± 36% +79.1% 151814 ± 5% numa-vmstat.node1.numa_other
10702 ± 18% -37.3% 6710 ± 16% numa-vmstat.node2.nr_slab_reclaimable
38337 ± 43% -43.0% 21857 ± 8% numa-vmstat.node2.nr_slab_unreclaimable
135697 ± 12% +17.9% 160016 ± 5% numa-vmstat.node2.numa_other
77071 ± 42% -41.7% 44935 ± 21% numa-vmstat.node3.nr_inactive_anon
199.00 ± 27% +121.4% 440.67 ± 39% numa-vmstat.node3.nr_page_table_pages
77070 ± 42% -41.7% 44935 ± 21% numa-vmstat.node3.nr_zone_inactive_anon
1317429 ± 6% +18.4% 1560077 ± 9% numa-vmstat.node3.numa_local
166193 -38.9% 101502 ± 50% numa-vmstat.node3.numa_other
51355399 ± 5% +17.9% 60523801 ± 9% sched_debug.cfs_rq:/.min_vruntime.avg
66657407 ± 6% +28.4% 85595963 ± 15% sched_debug.cfs_rq:/.min_vruntime.max
44178814 ± 7% +14.7% 50690341 ± 7% sched_debug.cfs_rq:/.min_vruntime.min
4396785 ± 33% +55.6% 6842280 ± 25% sched_debug.cfs_rq:/.min_vruntime.stddev
0.11 ± 3% -31.9% 0.07 ± 21% sched_debug.cfs_rq:/.nr_running.stddev
0.28 ± 45% +192.2% 0.83 ± 55% sched_debug.cfs_rq:/.removed.runnable_avg.avg
2.89 ± 48% +110.9% 6.10 ± 31% sched_debug.cfs_rq:/.removed.runnable_avg.stddev
0.28 ± 45% +192.2% 0.83 ± 55% sched_debug.cfs_rq:/.removed.util_avg.avg
2.89 ± 48% +110.9% 6.10 ± 31% sched_debug.cfs_rq:/.removed.util_avg.stddev
-6080241 +143.1% -14782494 sched_debug.cfs_rq:/.spread0.avg
-12219782 +95.2% -23848569 sched_debug.cfs_rq:/.spread0.min
4043652 ± 38% +61.7% 6539680 ± 23% sched_debug.cfs_rq:/.spread0.stddev
478.33 ± 6% +15.6% 552.77 ± 4% sched_debug.cfs_rq:/.util_avg.min
161.41 ± 2% -14.0% 138.76 ± 2% sched_debug.cfs_rq:/.util_avg.stddev
300408 ± 4% +14.6% 344155 ± 8% sched_debug.cpu.clock.avg
306855 ± 4% +14.2% 350489 ± 8% sched_debug.cpu.clock.max
291967 ± 4% +15.1% 336065 ± 8% sched_debug.cpu.clock.min
284821 ± 4% +14.1% 324960 ± 7% sched_debug.cpu.clock_task.avg
295707 ± 4% +14.4% 338410 ± 8% sched_debug.cpu.clock_task.max
263320 ± 5% +12.3% 295775 ± 7% sched_debug.cpu.clock_task.min
2763 ± 6% -17.0% 2292 ± 9% sched_debug.cpu.curr->pid.stddev
1290116 ± 33% -29.3% 911513 ± 6% sched_debug.cpu.max_idle_balance_cost.max
137888 ± 71% -61.3% 53408 ± 13% sched_debug.cpu.max_idle_balance_cost.stddev
4.07 ± 2% -18.4% 3.32 ± 13% sched_debug.cpu.nr_running.stddev
27664104 ± 3% +9.5% 30279647 ± 2% sched_debug.cpu.nr_switches.avg
39994582 ± 5% +10.6% 44251966 ± 3% sched_debug.cpu.nr_switches.max
16105574 ± 3% +12.0% 18041900 sched_debug.cpu.nr_switches.min
4467021 ± 6% +24.3% 5553088 ± 7% sched_debug.cpu.nr_switches.stddev
1.99 ± 17% -69.7% 0.60 ±105% sched_debug.cpu.nr_uninterruptible.avg
-101.08 +19.4% -120.74 sched_debug.cpu.nr_uninterruptible.min
291784 ± 4% +15.1% 335862 ± 8% sched_debug.cpu_clk
290911 ± 4% +15.2% 334987 ± 8% sched_debug.ktime
292141 ± 4% +15.1% 336220 ± 8% sched_debug.sched_clk
0.18 ± 49% -75.5% 0.05 ±133% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
9.03 ±110% -91.6% 0.76 ±130% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap
154.52 ±127% +4336.8% 6856 ±134% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
20267 ± 20% -66.3% 6840 ±108% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
23853 ± 13% +57.0% 37450 ± 17% perf-sched.total_wait_time.max.ms
133.73 ± 59% +87.1% 250.22 ± 34% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
42.92 ±141% +1554.6% 710.17 ± 73% perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
849.14 ± 13% +144.6% 2076 ± 17% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
38.67 ± 46% -79.3% 8.00 ±141% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown]
71.00 ± 26% -56.3% 31.00 ± 77% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
405.33 ± 13% -21.8% 317.00 ± 10% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork
20886 ± 20% +58.5% 33096 ± 27% perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
16704 ± 29% +122.2% 37109 ± 17% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork
0.30 ± 45% +2e+05% 588.96 ±141% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.__x64_sys_exit.do_syscall_64.entry_SYSCALL_64_after_hwframe
133.44 ± 59% +82.6% 243.63 ± 33% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
44.21 ±134% +1505.0% 709.59 ± 73% perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read
0.02 ± 12% +180.4% 0.05 ± 45% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_read.walk_component.link_path_walk
0.00 ±141% +17900.0% 0.36 ±134% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.perf_event_for_each_child._perf_ioctl
53.43 ± 16% +161.5% 139.70 ± 32% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity
0.01 ±141% +552.9% 0.07 ± 78% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.wp_page_copy.do_wp_page.__handle_mm_fault
545.69 ±117% -99.8% 1.04 ±139% perf-sched.wait_time.avg.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
776.66 ± 12% +161.0% 2026 ± 19% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork
0.55 ± 61% +5.7e+05% 3131 ±141% perf-sched.wait_time.max.ms.do_task_dead.do_exit.__x64_sys_exit.do_syscall_64.entry_SYSCALL_64_after_hwframe
20886 ± 20% +54.6% 32295 ± 26% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown]
23853 ± 13% +41.7% 33811 ± 23% perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown]
6067 ± 46% -73.3% 1618 ±133% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg
23050 ± 13% +38.6% 31938 ± 22% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.aa_sk_perm.security_socket_sendmsg.sock_sendmsg
0.03 ± 81% +133.7% 0.07 ± 42% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.terminate_walk.path_openat
0.00 ±141% +17900.0% 0.36 ±134% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.mutex_lock.perf_event_for_each_child._perf_ioctl
4272 ± 37% +76.3% 7531 ± 32% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity
0.03 ±141% +332.6% 0.14 ± 81% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.wp_page_copy.do_wp_page.__handle_mm_fault
570.36 ±140% -99.7% 1.65 ±141% perf-sched.wait_time.max.ms.rcu_gp_kthread.kthread.ret_from_fork
3612 ± 86% -99.9% 3.26 ±140% perf-sched.wait_time.max.ms.schedule_timeout.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
21679 ± 11% +52.3% 33025 ± 23% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
16704 ± 29% +122.2% 37109 ± 17% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork
37.10 ± 3% -2.4 34.73 perf-profile.calltrace.cycles-pp.__libc_read
36.20 ± 3% -2.3 33.91 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
35.20 ± 3% -2.1 33.07 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
35.08 ± 3% -2.1 32.97 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
33.84 ± 3% -2.0 31.88 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
0.64 -0.2 0.40 ± 70% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.unix_stream_read_generic
1.10 ± 12% -0.2 0.89 ± 4% perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
1.08 ± 12% -0.2 0.87 ± 4% perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
0.95 ± 15% -0.2 0.79 ± 7% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.__libc_read
0.92 ± 15% -0.1 0.78 ± 6% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.__libc_read
4.98 ± 20% +0.8 5.80 perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
8.27 ± 2% +1.0 9.27 ± 9% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.67 +1.0 9.68 ± 10% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
37.18 ± 3% -2.4 34.79 perf-profile.children.cycles-pp.__libc_read
2.75 ± 92% -2.3 0.48 ± 39% perf-profile.children.cycles-pp.update_cfs_group
1.36 ± 12% -0.2 1.11 ± 6% perf-profile.children.cycles-pp.alloc_skb_with_frags
1.32 ± 12% -0.2 1.09 ± 6% perf-profile.children.cycles-pp.__alloc_skb
0.19 ± 66% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.66 ± 2% -0.1 0.59 ± 2% perf-profile.children.cycles-pp.cpuacct_charge
0.29 ± 8% -0.1 0.23 ± 8% perf-profile.children.cycles-pp._copy_from_iter
0.55 ± 5% -0.1 0.49 ± 2% perf-profile.children.cycles-pp.set_next_entity
0.33 ± 5% -0.1 0.27 perf-profile.children.cycles-pp.security_socket_sendmsg
0.38 ± 4% -0.1 0.32 ± 2% perf-profile.children.cycles-pp.fsnotify
0.27 ± 8% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.48 ± 7% -0.0 0.43 ± 5% perf-profile.children.cycles-pp.___might_sleep
0.38 ± 3% -0.0 0.34 ± 5% perf-profile.children.cycles-pp.aa_sk_perm
0.24 ± 5% -0.0 0.19 ± 10% perf-profile.children.cycles-pp.sock_recvmsg
0.40 ± 6% -0.0 0.36 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.21 ± 4% -0.0 0.17 ± 11% perf-profile.children.cycles-pp.security_socket_recvmsg
0.18 ± 9% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.copyin
0.25 ± 6% -0.0 0.22 ± 5% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.15 ± 8% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.__check_heap_object
0.09 ± 19% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.__list_add_valid
0.14 ± 8% -0.0 0.12 ± 6% perf-profile.children.cycles-pp.__might_sleep
0.19 ± 6% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid
0.15 ± 3% -0.0 0.13 ± 10% perf-profile.children.cycles-pp.fsnotify_parent
0.08 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.wait_for_unix_gc
0.04 ± 71% +0.0 0.07 perf-profile.children.cycles-pp.__module_text_address
0.02 ±141% +0.0 0.06 ± 8% perf-profile.children.cycles-pp.perf_swevent_event
0.31 ± 6% +0.0 0.35 ± 3% perf-profile.children.cycles-pp.task_tick_fair
0.02 ±141% +0.0 0.06 perf-profile.children.cycles-pp.__perf_event_account_interrupt
0.06 ± 71% +0.0 0.10 perf-profile.children.cycles-pp.is_module_text_address
0.34 ± 6% +0.1 0.40 ± 2% perf-profile.children.cycles-pp.scheduler_tick
0.15 ± 44% +0.1 0.23 ± 7% perf-profile.children.cycles-pp.perf_tp_event_match
0.41 ± 6% +0.1 0.49 ± 6% perf-profile.children.cycles-pp.update_process_times
0.41 ± 6% +0.1 0.49 ± 5% perf-profile.children.cycles-pp.tick_sched_handle
0.44 ± 6% +0.1 0.53 ± 6% perf-profile.children.cycles-pp.tick_sched_timer
0.21 ± 33% +0.1 0.32 ± 15% perf-profile.children.cycles-pp.in_task_stack
0.49 ± 4% +0.1 0.61 ± 8% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.30 ± 35% +0.1 0.45 ± 11% perf-profile.children.cycles-pp.get_stack_info
0.69 +0.3 0.94 ± 16% perf-profile.children.cycles-pp.hrtimer_interrupt
0.69 ± 2% +0.3 0.95 ± 16% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.76 +0.3 1.03 ± 16% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.86 ± 3% +0.3 1.13 ± 15% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
2.74 ± 92% -2.3 0.47 ± 39% perf-profile.self.cycles-pp.update_cfs_group
0.32 ± 52% -0.2 0.13 ± 6% perf-profile.self.cycles-pp.___perf_sw_event
0.37 ± 27% -0.1 0.26 ± 13% perf-profile.self.cycles-pp.__ksize
0.66 ± 2% -0.1 0.59 ± 2% perf-profile.self.cycles-pp.cpuacct_charge
0.32 ± 6% -0.1 0.25 ± 3% perf-profile.self.cycles-pp.common_file_perm
0.35 ± 3% -0.1 0.30 perf-profile.self.cycles-pp.fsnotify
0.14 ± 23% -0.0 0.10 ± 12% perf-profile.self.cycles-pp.mutex_unlock
0.16 ± 10% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.aa_sk_perm
0.18 ± 4% -0.0 0.15 ± 5% perf-profile.self.cycles-pp.reweight_entity
0.16 ± 8% -0.0 0.13 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.13 ± 9% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.__alloc_skb
0.14 -0.0 0.11 ± 8% perf-profile.self.cycles-pp.fsnotify_parent
0.12 ± 6% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.__libc_read
0.24 ± 5% -0.0 0.22 ± 5% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.16 ± 7% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.14 ± 8% -0.0 0.12 perf-profile.self.cycles-pp.check_preempt_wakeup
0.14 ± 8% -0.0 0.12 ± 6% perf-profile.self.cycles-pp.__check_heap_object
0.12 ± 10% -0.0 0.10 ± 8% perf-profile.self.cycles-pp.__might_sleep
0.08 ± 12% -0.0 0.06 ± 8% perf-profile.self.cycles-pp._copy_from_iter
0.14 ± 5% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.enqueue_entity
0.08 ± 10% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.unix_stream_recvmsg
0.07 ± 18% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.autoremove_wake_function
0.05 ± 71% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.perf_misc_flags
0.04 ± 71% +0.0 0.09 ± 10% perf-profile.self.cycles-pp.perf_swevent_overflow
0.34 ± 6% +0.0 0.38 ± 7% perf-profile.self.cycles-pp.vfs_write
0.07 ± 71% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.perf_event_pid_type
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.__perf_event_account_interrupt
0.14 ± 46% +0.1 0.20 ± 7% perf-profile.self.cycles-pp.perf_tp_event_match
0.16 ± 40% +0.1 0.24 ± 7% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.41 ± 21% +0.1 0.51 ± 4% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.21 ± 33% +0.1 0.31 ± 17% perf-profile.self.cycles-pp.in_task_stack
0.90 ± 6% +0.1 1.04 ± 4% perf-profile.self.cycles-pp.perf_tp_event
0.26 ± 32% +0.2 0.42 ± 14% perf-profile.self.cycles-pp.__perf_event_overflow
0.32 ± 43% +0.2 0.49 ± 2% perf-profile.self.cycles-pp.perf_prepare_sample
0.58 ± 28% +0.2 0.83 ± 2% perf-profile.self.cycles-pp.perf_trace_sched_switch
11383 ± 69% +426.8% 59963 ± 45% interrupts.39:PCI-MSI.524290-edge.eth0-TxRx-1
5445 ± 40% +174.1% 14925 ± 59% interrupts.41:PCI-MSI.524292-edge.eth0-TxRx-3
2457907 ± 11% -22.6% 1901728 interrupts.CPU1.RES:Rescheduling_interrupts
2641257 -17.6% 2177600 ± 6% interrupts.CPU104.RES:Rescheduling_interrupts
11383 ± 69% +426.8% 59963 ± 45% interrupts.CPU11.39:PCI-MSI.524290-edge.eth0-TxRx-1
5160330 ± 8% -20.9% 4079569 ± 16% interrupts.CPU110.CAL:Function_call_interrupts
4480782 ± 10% +16.6% 5223363 ± 10% interrupts.CPU112.CAL:Function_call_interrupts
1993851 ± 17% +30.8% 2607702 ± 6% interrupts.CPU114.RES:Rescheduling_interrupts
1920162 ± 20% +43.8% 2760707 ± 14% interrupts.CPU115.RES:Rescheduling_interrupts
1838014 ± 19% +29.9% 2387620 ± 13% interrupts.CPU118.RES:Rescheduling_interrupts
5778 ± 35% +50.6% 8699 interrupts.CPU121.NMI:Non-maskable_interrupts
5778 ± 35% +50.6% 8699 interrupts.CPU121.PMI:Performance_monitoring_interrupts
5770 ± 35% +25.7% 7251 ± 28% interrupts.CPU124.NMI:Non-maskable_interrupts
5770 ± 35% +25.7% 7251 ± 28% interrupts.CPU124.PMI:Performance_monitoring_interrupts
5445 ± 40% +174.1% 14925 ± 59% interrupts.CPU13.41:PCI-MSI.524292-edge.eth0-TxRx-3
5188909 ± 3% +26.8% 6578819 ± 15% interrupts.CPU130.CAL:Function_call_interrupts
1741448 ± 13% +28.9% 2244772 ± 8% interrupts.CPU134.RES:Rescheduling_interrupts
4870246 ± 14% +23.2% 6002115 ± 13% interrupts.CPU135.CAL:Function_call_interrupts
2104688 ± 11% +22.1% 2570605 ± 9% interrupts.CPU138.RES:Rescheduling_interrupts
5567970 ± 5% -15.6% 4699013 ± 4% interrupts.CPU141.CAL:Function_call_interrupts
6426481 ± 17% -28.6% 4585671 ± 12% interrupts.CPU144.CAL:Function_call_interrupts
6230112 ± 12% -20.6% 4948034 ± 11% interrupts.CPU145.CAL:Function_call_interrupts
1567730 ± 26% +46.8% 2302159 ± 10% interrupts.CPU146.RES:Rescheduling_interrupts
1846928 ± 8% +12.3% 2074112 interrupts.CPU148.RES:Rescheduling_interrupts
1814816 ± 6% +11.5% 2023890 ± 5% interrupts.CPU149.RES:Rescheduling_interrupts
6912968 ± 13% -32.8% 4642666 ± 21% interrupts.CPU151.CAL:Function_call_interrupts
2319343 ± 9% -19.0% 1877846 ± 7% interrupts.CPU152.RES:Rescheduling_interrupts
1822993 ± 12% +43.2% 2610289 ± 16% interrupts.CPU153.RES:Rescheduling_interrupts
5838607 ± 5% -18.7% 4744873 ± 17% interrupts.CPU158.CAL:Function_call_interrupts
5926829 ± 10% -16.9% 4927464 ± 9% interrupts.CPU161.CAL:Function_call_interrupts
1682857 ± 19% +36.2% 2292749 ± 11% interrupts.CPU164.RES:Rescheduling_interrupts
4116921 ± 18% +66.9% 6869135 ± 32% interrupts.CPU171.CAL:Function_call_interrupts
4142580 +60.6% 6652336 ± 16% interrupts.CPU172.CAL:Function_call_interrupts
4502353 ± 18% +46.8% 6610734 ± 8% interrupts.CPU173.CAL:Function_call_interrupts
4659039 ± 14% +46.3% 6818440 ± 34% interrupts.CPU175.CAL:Function_call_interrupts
4447231 ± 9% +66.2% 7389442 ± 21% interrupts.CPU176.CAL:Function_call_interrupts
4255423 ± 13% +56.4% 6654582 ± 15% interrupts.CPU177.CAL:Function_call_interrupts
4319938 ± 10% +33.7% 5774236 ± 10% interrupts.CPU178.CAL:Function_call_interrupts
4203332 ± 9% +74.5% 7335115 ± 22% interrupts.CPU179.CAL:Function_call_interrupts
4614871 ± 10% +51.8% 7005269 ± 23% interrupts.CPU180.CAL:Function_call_interrupts
2181783 ± 3% -19.7% 1751755 ± 4% interrupts.CPU180.RES:Rescheduling_interrupts
4050520 ± 11% +75.9% 7123398 ± 15% interrupts.CPU182.CAL:Function_call_interrupts
1699459 ± 3% +32.3% 2249005 ± 16% interrupts.CPU182.RES:Rescheduling_interrupts
4990167 ± 5% +40.9% 7029004 ± 12% interrupts.CPU183.CAL:Function_call_interrupts
5150715 ± 20% +29.7% 6678194 ± 5% interrupts.CPU185.CAL:Function_call_interrupts
4503281 ± 8% +59.8% 7197423 ± 13% interrupts.CPU187.CAL:Function_call_interrupts
4827738 ± 14% +26.0% 6081651 ± 16% interrupts.CPU190.CAL:Function_call_interrupts
4726248 ± 8% -23.2% 3631004 ± 20% interrupts.CPU2.CAL:Function_call_interrupts
2067404 ± 9% -17.1% 1713473 ± 11% interrupts.CPU2.RES:Rescheduling_interrupts
1712460 ± 7% +19.9% 2053835 interrupts.CPU22.RES:Rescheduling_interrupts
3238643 ± 22% +27.6% 4131745 ± 12% interrupts.CPU29.CAL:Function_call_interrupts
1607899 ± 4% +28.7% 2069474 ± 14% interrupts.CPU29.RES:Rescheduling_interrupts
3601982 ± 8% +15.5% 4159215 ± 10% interrupts.CPU31.CAL:Function_call_interrupts
3801797 ± 4% +18.1% 4488433 ± 12% interrupts.CPU32.CAL:Function_call_interrupts
3374468 ± 10% +23.8% 4176507 ± 2% interrupts.CPU33.CAL:Function_call_interrupts
1753123 ± 4% +13.8% 1994342 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts
1901796 +14.2% 2171182 ± 4% interrupts.CPU35.RES:Rescheduling_interrupts
2424585 ± 8% -16.6% 2022154 ± 2% interrupts.CPU45.RES:Rescheduling_interrupts
3750015 ± 14% +29.2% 4843627 ± 15% interrupts.CPU46.CAL:Function_call_interrupts
5556270 ± 21% -36.8% 3509724 ± 28% interrupts.CPU48.CAL:Function_call_interrupts
5711601 ± 15% -33.8% 3780223 ± 22% interrupts.CPU49.CAL:Function_call_interrupts
1613828 ± 18% +32.2% 2133584 ± 12% interrupts.CPU49.RES:Rescheduling_interrupts
1457715 ± 18% +43.2% 2087070 interrupts.CPU52.RES:Rescheduling_interrupts
1789983 ± 10% +23.7% 2214022 ± 16% interrupts.CPU54.RES:Rescheduling_interrupts
5662000 ± 26% -37.4% 3542511 ± 13% interrupts.CPU57.CAL:Function_call_interrupts
5269193 ± 12% -28.0% 3796389 ± 22% interrupts.CPU60.CAL:Function_call_interrupts
2215937 ± 10% -15.8% 1866080 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts
4959625 ± 2% -22.4% 3846220 ± 17% interrupts.CPU63.CAL:Function_call_interrupts
2357629 ± 13% -27.5% 1708837 ± 18% interrupts.CPU63.RES:Rescheduling_interrupts
5291221 ± 10% -24.8% 3977451 ± 8% interrupts.CPU65.CAL:Function_call_interrupts
1630238 ± 26% +42.5% 2322835 ± 8% interrupts.CPU68.RES:Rescheduling_interrupts
5404145 ± 5% -25.2% 4044518 ± 18% interrupts.CPU69.CAL:Function_call_interrupts
4311698 ± 5% +43.0% 6164696 ± 13% interrupts.CPU72.CAL:Function_call_interrupts
3677831 ± 9% +70.6% 6273317 ± 29% interrupts.CPU73.CAL:Function_call_interrupts
3612623 ± 10% +54.6% 5583789 ± 22% interrupts.CPU74.CAL:Function_call_interrupts
3856325 ± 4% +51.3% 5834147 ± 15% interrupts.CPU75.CAL:Function_call_interrupts
1645387 ± 3% +27.5% 2098281 ± 14% interrupts.CPU75.RES:Rescheduling_interrupts
3739398 ± 11% +66.8% 6238004 ± 29% interrupts.CPU76.CAL:Function_call_interrupts
1598254 ± 19% +29.3% 2066964 ± 9% interrupts.CPU77.RES:Rescheduling_interrupts
3350555 ± 11% +52.9% 5124400 ± 15% interrupts.CPU80.CAL:Function_call_interrupts
3574937 ± 12% +46.1% 5224402 ± 8% interrupts.CPU81.CAL:Function_call_interrupts
3881925 ± 20% +38.7% 5385256 ± 23% interrupts.CPU82.CAL:Function_call_interrupts
3614411 ± 14% +65.7% 5990688 ± 16% interrupts.CPU83.CAL:Function_call_interrupts
3972826 ± 16% +31.4% 5218713 ± 6% interrupts.CPU84.CAL:Function_call_interrupts
4000216 ± 6% +52.0% 6079848 ± 24% interrupts.CPU85.CAL:Function_call_interrupts
3764192 ± 11% +42.2% 5353364 ± 17% interrupts.CPU86.CAL:Function_call_interrupts
3629453 ± 2% +54.7% 5613811 ± 20% interrupts.CPU87.CAL:Function_call_interrupts
3678632 ± 10% +70.2% 6260271 ± 26% interrupts.CPU88.CAL:Function_call_interrupts
3830790 ± 11% +63.4% 6260528 ± 25% interrupts.CPU90.CAL:Function_call_interrupts
5784 ± 35% +50.4% 8699 interrupts.CPU91.NMI:Non-maskable_interrupts
5784 ± 35% +50.4% 8699 interrupts.CPU91.PMI:Performance_monitoring_interrupts
3341401 ± 7% +57.7% 5268798 ± 8% interrupts.CPU92.CAL:Function_call_interrupts
3696549 ± 3% +86.3% 6885440 ± 27% interrupts.CPU93.CAL:Function_call_interrupts
5786 ± 35% +50.3% 8698 interrupts.CPU93.NMI:Non-maskable_interrupts
5786 ± 35% +50.3% 8698 interrupts.CPU93.PMI:Performance_monitoring_interrupts
4199119 ± 8% +39.0% 5835061 ± 10% interrupts.CPU94.CAL:Function_call_interrupts
3820035 ± 6% +57.8% 6027549 ± 28% interrupts.CPU95.CAL:Function_call_interrupts
5790 ± 35% +50.1% 8694 interrupts.CPU95.NMI:Non-maskable_interrupts
5790 ± 35% +50.1% 8694 interrupts.CPU95.PMI:Performance_monitoring_interrupts
1808381 ± 6% +26.4% 2286065 ± 11% interrupts.CPU95.RES:Rescheduling_interrupts
99695 ± 8% +27.7% 127348 ± 12% softirqs.CPU0.RCU
102168 ± 7% +26.6% 129395 ± 11% softirqs.CPU1.RCU
101438 ± 6% +25.1% 126885 ± 10% softirqs.CPU10.RCU
101352 ± 7% +24.3% 126007 ± 10% softirqs.CPU100.RCU
101061 ± 8% +24.9% 126249 ± 11% softirqs.CPU102.RCU
100479 ± 8% +25.1% 125672 ± 10% softirqs.CPU103.RCU
101891 ± 7% +23.5% 125847 ± 11% softirqs.CPU104.RCU
100756 ± 7% +24.3% 125247 ± 10% softirqs.CPU105.RCU
100557 ± 7% +23.9% 124600 ± 10% softirqs.CPU106.RCU
100714 ± 7% +25.7% 126571 ± 10% softirqs.CPU107.RCU
101251 ± 7% +24.3% 125831 ± 10% softirqs.CPU108.RCU
100308 ± 7% +25.5% 125861 ± 10% softirqs.CPU109.RCU
5798 ± 68% +429.4% 30699 ± 46% softirqs.CPU11.NET_RX
101135 ± 7% +27.1% 128567 ± 10% softirqs.CPU11.RCU
100161 ± 7% +24.2% 124410 ± 10% softirqs.CPU110.RCU
98850 ± 11% +27.5% 126012 ± 10% softirqs.CPU111.RCU
104203 ± 6% +26.4% 131712 ± 11% softirqs.CPU112.RCU
101400 ± 5% +29.1% 130884 ± 10% softirqs.CPU113.RCU
103374 ± 6% +20.8% 124837 ± 8% softirqs.CPU114.RCU
104312 ± 6% +24.8% 130182 ± 11% softirqs.CPU115.RCU
100093 ± 6% +28.3% 128418 ± 14% softirqs.CPU116.RCU
103618 ± 7% +25.7% 130214 ± 11% softirqs.CPU117.RCU
103544 ± 8% +26.1% 130595 ± 11% softirqs.CPU118.RCU
104245 ± 6% +26.6% 131973 ± 11% softirqs.CPU119.RCU
102211 ± 7% +25.7% 128480 ± 11% softirqs.CPU12.RCU
104976 ± 4% +28.1% 134429 ± 9% softirqs.CPU120.RCU
105246 ± 4% +28.3% 135026 ± 9% softirqs.CPU121.RCU
105073 ± 4% +27.3% 133799 ± 9% softirqs.CPU122.RCU
105312 ± 4% +26.5% 133244 ± 9% softirqs.CPU123.RCU
104797 ± 4% +27.6% 133755 ± 8% softirqs.CPU124.RCU
104474 ± 4% +27.4% 133097 ± 8% softirqs.CPU125.RCU
104492 ± 4% +29.2% 135051 ± 9% softirqs.CPU126.RCU
105827 ± 2% +26.9% 134277 ± 9% softirqs.CPU127.RCU
101849 ± 3% +27.4% 129782 ± 7% softirqs.CPU128.RCU
101012 ± 3% +28.8% 130053 ± 7% softirqs.CPU129.RCU
2774 ± 40% +174.4% 7614 ± 57% softirqs.CPU13.NET_RX
101485 ± 7% +25.3% 127149 ± 10% softirqs.CPU13.RCU
100533 ± 3% +27.2% 127897 ± 7% softirqs.CPU130.RCU
101081 ± 3% +27.8% 129222 ± 7% softirqs.CPU131.RCU
101450 ± 3% +28.4% 130217 ± 7% softirqs.CPU132.RCU
101700 ± 3% +27.7% 129857 ± 7% softirqs.CPU133.RCU
100851 ± 3% +27.5% 128605 ± 8% softirqs.CPU134.RCU
101043 ± 4% +27.1% 128377 ± 7% softirqs.CPU135.RCU
101163 ± 3% +27.8% 129269 ± 7% softirqs.CPU136.RCU
101350 ± 3% +28.8% 130556 ± 8% softirqs.CPU137.RCU
101569 ± 4% +28.2% 130187 ± 7% softirqs.CPU138.RCU
103024 ± 3% +26.8% 130598 ± 7% softirqs.CPU139.RCU
101217 ± 6% +26.6% 128113 ± 10% softirqs.CPU14.RCU
101171 ± 3% +28.0% 129543 ± 8% softirqs.CPU140.RCU
100983 ± 3% +27.6% 128904 ± 8% softirqs.CPU141.RCU
100777 ± 4% +27.7% 128651 ± 8% softirqs.CPU142.RCU
101548 ± 3% +26.1% 128069 ± 6% softirqs.CPU143.RCU
102292 ± 4% +28.7% 131622 ± 7% softirqs.CPU144.RCU
101796 ± 3% +29.3% 131646 ± 8% softirqs.CPU145.RCU
101098 ± 3% +28.6% 130051 ± 8% softirqs.CPU146.RCU
101267 ± 3% +28.2% 129816 ± 8% softirqs.CPU147.RCU
101847 ± 4% +27.7% 130042 ± 8% softirqs.CPU148.RCU
100743 ± 3% +28.3% 129222 ± 7% softirqs.CPU149.RCU
98982 ± 10% +28.7% 127411 ± 10% softirqs.CPU15.RCU
100636 ± 4% +29.0% 129827 ± 8% softirqs.CPU150.RCU
101113 ± 5% +30.0% 131420 ± 8% softirqs.CPU151.RCU
101505 ± 4% +29.0% 130930 ± 7% softirqs.CPU152.RCU
100566 ± 4% +30.3% 131071 ± 8% softirqs.CPU153.RCU
101747 ± 5% +27.9% 130127 ± 7% softirqs.CPU154.RCU
99882 ± 3% +28.1% 127990 ± 8% softirqs.CPU155.RCU
102357 ± 4% +28.3% 131323 ± 7% softirqs.CPU156.RCU
101490 ± 4% +29.4% 131298 ± 8% softirqs.CPU157.RCU
101391 ± 4% +28.5% 130291 ± 7% softirqs.CPU158.RCU
101583 ± 3% +28.3% 130345 ± 8% softirqs.CPU159.RCU
104519 ± 6% +25.9% 131595 ± 11% softirqs.CPU16.RCU
105238 ± 3% +30.4% 137272 ± 8% softirqs.CPU160.RCU
105282 ± 4% +29.8% 136612 ± 9% softirqs.CPU161.RCU
106084 ± 4% +28.8% 136592 ± 9% softirqs.CPU162.RCU
105736 ± 4% +27.8% 135088 ± 9% softirqs.CPU163.RCU
104681 ± 4% +30.2% 136271 ± 8% softirqs.CPU164.RCU
105572 ± 3% +27.8% 134963 ± 8% softirqs.CPU165.RCU
105612 ± 4% +29.6% 136902 ± 8% softirqs.CPU166.RCU
106257 ± 3% +28.6% 136631 ± 9% softirqs.CPU167.RCU
102940 ± 3% +18.9% 122351 ± 5% softirqs.CPU168.RCU
102433 ± 4% +19.3% 122208 ± 5% softirqs.CPU169.RCU
101963 ± 4% +28.2% 130705 ± 10% softirqs.CPU17.RCU
103129 ± 4% +19.5% 123216 ± 5% softirqs.CPU170.RCU
102988 ± 3% +19.9% 123495 ± 4% softirqs.CPU171.RCU
102638 ± 3% +19.8% 122960 ± 5% softirqs.CPU172.RCU
102750 ± 4% +20.0% 123326 ± 4% softirqs.CPU173.RCU
103262 ± 3% +19.5% 123363 ± 6% softirqs.CPU174.RCU
102636 ± 4% +19.1% 122231 ± 4% softirqs.CPU175.RCU
103507 ± 4% +14.2% 118205 ± 3% softirqs.CPU176.RCU
102388 ± 3% +14.7% 117390 ± 3% softirqs.CPU177.RCU
102423 ± 3% +15.3% 118067 ± 4% softirqs.CPU179.RCU
103346 ± 6% +23.0% 127133 ± 8% softirqs.CPU18.RCU
103193 ± 3% +15.3% 118961 ± 3% softirqs.CPU181.RCU
103182 ± 3% +14.9% 118546 ± 4% softirqs.CPU182.RCU
102714 ± 4% +15.2% 118323 ± 4% softirqs.CPU183.RCU
103227 ± 3% +14.8% 118493 ± 3% softirqs.CPU184.RCU
102946 ± 4% +15.3% 118676 ± 3% softirqs.CPU186.RCU
102318 ± 4% +15.6% 118299 ± 3% softirqs.CPU188.RCU
104098 ± 6% +24.7% 129778 ± 10% softirqs.CPU19.RCU
102541 ± 3% +15.0% 117893 ± 3% softirqs.CPU190.RCU
101042 ± 7% +28.0% 129341 ± 10% softirqs.CPU2.RCU
101819 ± 5% +26.2% 128473 ± 13% softirqs.CPU20.RCU
103710 ± 5% +26.2% 130919 ± 11% softirqs.CPU21.RCU
103462 ± 6% +26.2% 130556 ± 11% softirqs.CPU22.RCU
104698 ± 6% +23.6% 129449 ± 11% softirqs.CPU23.RCU
105912 ± 4% +28.9% 136526 ± 9% softirqs.CPU24.RCU
106915 ± 5% +28.0% 136862 ± 9% softirqs.CPU25.RCU
108194 ± 5% +25.2% 135427 ± 8% softirqs.CPU26.RCU
106435 ± 5% +26.4% 134565 ± 9% softirqs.CPU27.RCU
106432 ± 5% +27.1% 135282 ± 9% softirqs.CPU28.RCU
106762 ± 5% +26.4% 134947 ± 9% softirqs.CPU29.RCU
101073 ± 7% +26.0% 127315 ± 11% softirqs.CPU3.RCU
106558 ± 4% +28.3% 136717 ± 9% softirqs.CPU30.RCU
107555 ± 5% +26.0% 135530 ± 10% softirqs.CPU31.RCU
103366 ± 3% +28.3% 132635 ± 8% softirqs.CPU32.RCU
102621 ± 4% +28.4% 131744 ± 6% softirqs.CPU33.RCU
102563 ± 3% +26.9% 130144 ± 7% softirqs.CPU34.RCU
102718 ± 3% +26.7% 130101 ± 7% softirqs.CPU35.RCU
102256 ± 3% +28.2% 131054 ± 7% softirqs.CPU36.RCU
102832 ± 2% +28.2% 131813 ± 7% softirqs.CPU37.RCU
103243 ± 3% +26.9% 131040 ± 8% softirqs.CPU38.RCU
103107 ± 3% +27.8% 131771 ± 7% softirqs.CPU39.RCU
100677 ± 7% +26.5% 127350 ± 10% softirqs.CPU4.RCU
102675 ± 3% +27.9% 131327 ± 7% softirqs.CPU40.RCU
102932 ± 3% +27.3% 131012 ± 8% softirqs.CPU41.RCU
101673 ± 3% +27.3% 129440 ± 7% softirqs.CPU42.RCU
103102 ± 3% +28.0% 131926 ± 8% softirqs.CPU43.RCU
102486 ± 3% +27.9% 131040 ± 8% softirqs.CPU44.RCU
102649 ± 3% +26.9% 130231 ± 8% softirqs.CPU45.RCU
101143 ± 3% +28.5% 129932 ± 8% softirqs.CPU46.RCU
101886 ± 3% +25.0% 127395 ± 6% softirqs.CPU47.RCU
102679 ± 3% +28.8% 132211 ± 8% softirqs.CPU48.RCU
102383 ± 4% +29.1% 132151 ± 8% softirqs.CPU49.RCU
101739 ± 4% +29.5% 131737 ± 9% softirqs.CPU50.RCU
101851 ± 3% +28.2% 130554 ± 8% softirqs.CPU51.RCU
100790 ± 4% +28.6% 129635 ± 8% softirqs.CPU52.RCU
102025 ± 3% +29.5% 132075 ± 7% softirqs.CPU53.RCU
101675 ± 4% +29.0% 131203 ± 7% softirqs.CPU54.RCU
102229 ± 4% +28.5% 131337 ± 8% softirqs.CPU55.RCU
101345 ± 4% +28.8% 130535 ± 8% softirqs.CPU56.RCU
101302 ± 3% +29.9% 131631 ± 8% softirqs.CPU57.RCU
102543 ± 4% +26.3% 129467 ± 6% softirqs.CPU58.RCU
101876 ± 3% +28.0% 130376 ± 8% softirqs.CPU59.RCU
100954 ± 7% +26.2% 127397 ± 11% softirqs.CPU6.RCU
102808 ± 3% +28.6% 132232 ± 8% softirqs.CPU60.RCU
102230 ± 4% +28.0% 130855 ± 9% softirqs.CPU61.RCU
101560 ± 3% +27.6% 129585 ± 8% softirqs.CPU62.RCU
101792 ± 4% +28.0% 130286 ± 9% softirqs.CPU63.RCU
105773 ± 3% +30.0% 137481 ± 8% softirqs.CPU64.RCU
105212 ± 4% +29.7% 136426 ± 9% softirqs.CPU65.RCU
107146 ± 3% +29.6% 138847 ± 9% softirqs.CPU66.RCU
106932 ± 3% +30.1% 139150 ± 9% softirqs.CPU67.RCU
106536 ± 3% +29.8% 138241 ± 8% softirqs.CPU68.RCU
105525 ± 3% +30.4% 137607 ± 9% softirqs.CPU69.RCU
101492 ± 7% +25.4% 127288 ± 10% softirqs.CPU7.RCU
105803 ± 3% +29.1% 136623 ± 9% softirqs.CPU70.RCU
105655 ± 4% +30.5% 137861 ± 9% softirqs.CPU71.RCU
103655 ± 3% +21.4% 125865 ± 6% softirqs.CPU72.RCU
103901 ± 4% +20.2% 124850 ± 6% softirqs.CPU73.RCU
103789 ± 4% +21.1% 125669 ± 5% softirqs.CPU74.RCU
103170 ± 3% +21.9% 125769 ± 5% softirqs.CPU75.RCU
103156 ± 3% +21.3% 125098 ± 5% softirqs.CPU76.RCU
102168 ± 3% +22.5% 125114 ± 5% softirqs.CPU77.RCU
102852 ± 3% +20.8% 124219 ± 5% softirqs.CPU78.RCU
103777 ± 4% +20.8% 125375 ± 5% softirqs.CPU79.RCU
101313 ± 7% +26.6% 128252 ± 12% softirqs.CPU8.RCU
102620 ± 4% +15.7% 118694 ± 4% softirqs.CPU80.RCU
101557 ± 3% +15.8% 117652 ± 4% softirqs.CPU81.RCU
101501 ± 4% +16.0% 117701 ± 3% softirqs.CPU82.RCU
102729 ± 4% +15.2% 118338 ± 4% softirqs.CPU83.RCU
101540 ± 3% +16.5% 118250 ± 4% softirqs.CPU86.RCU
101607 ± 3% +15.5% 117332 ± 4% softirqs.CPU87.RCU
102737 ± 4% +15.2% 118336 ± 3% softirqs.CPU89.RCU
100455 ± 7% +23.6% 124154 ± 8% softirqs.CPU9.RCU
102382 ± 4% +15.5% 118301 ± 4% softirqs.CPU90.RCU
102257 ± 4% +17.7% 120341 ± 3% softirqs.CPU92.RCU
101809 ± 4% +15.9% 118027 ± 4% softirqs.CPU93.RCU
5986 ± 5% +36.4% 8162 ± 18% softirqs.CPU93.SCHED
102201 ± 3% +16.1% 118678 ± 4% softirqs.CPU94.RCU
102079 ± 4% +15.5% 117850 ± 3% softirqs.CPU95.RCU
99893 ± 7% +26.9% 126781 ± 10% softirqs.CPU96.RCU
101943 ± 8% +22.4% 124784 ± 10% softirqs.CPU97.RCU
100354 ± 7% +25.6% 126020 ± 11% softirqs.CPU98.RCU
101113 ± 7% +25.3% 126708 ± 11% softirqs.CPU99.RCU
16498 ± 32% +250.1% 57762 ± 20% softirqs.NET_RX
19715004 ± 4% +24.7% 24582787 ± 7% softirqs.RCU
1246365 ± 2% +12.5% 1401902 ± 4% softirqs.SCHED
126945 ± 2% +18.0% 149827 ± 5% softirqs.TIMER
***************************************************************************************************
lkp-cfl-e1: 16 threads 1 sockets Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/30%/debian-10.4-x86_64-20200603.cgz/300s/lkp-cfl-e1/shell1/unixbench/0xde
commit:
bf4f059954 ("mm: memcg/slab: obj_cgroup API")
286e04b8ed ("mm: memcg/slab: allocate obj_cgroups for non-root slab pages")
bf4f059954dcb221 286e04b8ed7a04279ae277f0f02
---------------- ---------------------------
%stddev %change %stddev
\ | \
1.372e+08 ± 38% -43.9% 77022150 ± 12% cpuidle.C1.time
1933835 ± 9% -12.9% 1683420 ± 9% cpuidle.C1.usage
49.39 +8.1% 53.40 ± 3% perf-stat.i.MPKI
2.46 +0.2 2.62 perf-stat.i.branch-miss-rate%
922.24 ± 9% +48.1% 1365 ± 36% sched_debug.cfs_rq:/.load_avg.max
257.37 ± 7% +46.4% 376.78 ± 35% sched_debug.cfs_rq:/.load_avg.stddev
1127 ± 5% -8.8% 1027 ± 5% sched_debug.cfs_rq:/.runnable_avg.max
1120 ± 5% -9.0% 1019 ± 5% sched_debug.cfs_rq:/.util_avg.max
0.00 ± 9% -23.0% 0.00 ± 20% sched_debug.cpu.next_balance.stddev
0.59 ± 3% +0.1 0.64 ± 2% perf-profile.calltrace.cycles-pp.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64
6.20 ± 6% +0.3 6.55 ± 6% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
0.00 +0.5 0.52 perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect
0.18 ± 6% -0.0 0.15 ± 5% perf-profile.children.cycles-pp.security_mmap_file
0.17 ± 6% -0.0 0.15 ± 11% perf-profile.children.cycles-pp.anon_vma_clone
0.12 ± 8% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.__put_user_8
0.13 ± 7% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.cp_new_stat
0.12 ± 9% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.up_read
0.09 ± 5% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.close@plt
0.24 ± 3% +0.0 0.26 ± 3% perf-profile.children.cycles-pp.open64
0.18 ± 5% +0.0 0.20 ± 6% perf-profile.children.cycles-pp.wmemchr
0.37 ± 5% +0.1 0.43 ± 2% perf-profile.children.cycles-pp.__close
0.96 ± 2% +0.1 1.04 ± 5% perf-profile.children.cycles-pp.__vma_adjust
0.12 ± 9% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.up_read
0.22 ± 5% -0.0 0.20 ± 7% perf-profile.self.cycles-pp.mark_page_accessed
0.10 ± 10% -0.0 0.08 ± 10% perf-profile.self.cycles-pp.__put_user_8
0.10 ± 11% -0.0 0.08 ± 10% perf-profile.self.cycles-pp.do_fault
0.02 ±141% +0.0 0.06 ± 8% perf-profile.self.cycles-pp.acct_collect
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation
Thanks,
Oliver Sang
View attachment "config-5.8.0-09681-g286e04b8ed7a0" of type "text/plain" (169747 bytes)
View attachment "job-script" of type "text/plain" (7942 bytes)
View attachment "job.yaml" of type "text/plain" (5401 bytes)
View attachment "reproduce" of type "text/plain" (335 bytes)
Powered by blists - more mailing lists