lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20191114025910.GA24535@xsang-OptiPlex-9020>
Date:   Thu, 14 Nov 2019 10:59:10 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Alexander Potapenko <glider@...gle.com>
Cc:     Linus Torvalds <torvalds@...ux-foundation.org>,
        Kees Cook <keescook@...omium.org>,
        Christoph Lameter <cl@...ux.com>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        "Serge E. Hallyn" <serge@...lyn.com>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Kostya Serebryany <kcc@...gle.com>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Sandeep Patil <sspatil@...roid.com>,
        Laura Abbott <labbott@...hat.com>,
        Randy Dunlap <rdunlap@...radead.org>,
        Jann Horn <jannh@...gle.com>,
        Mark Rutland <mark.rutland@....com>,
        Marco Elver <elver@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: [mm]  6471384af2:  hackbench.throughput -4.9% regression

Greeting,

FYI, we noticed a -4.9% regression of hackbench.throughput due to commit:


commit: 6471384af2a6530696fc0203bafe4de41a23c9ef ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: hackbench
on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
with following parameters:

	nr_threads: 100%
	mode: process
	ipc: socket
	cpufreq_governor: performance
	ucode: 0xb8

test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sched/cfs-scheduler/hackbench.c

In addition to that, the commit also has significant impact on the following tests:

+------------------+-------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -4.3% regression                  |
| test machine     | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory |
| test parameters  | cpufreq_governor=performance                                      |
|                  | mode=process                                                      |
|                  | nr_threads=100%                                                   |
|                  | ucode=0xb8                                                        |
+------------------+-------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
  gcc-7/performance/socket/x86_64-rhel-7.2-clear_lck_7595/process/100%/clear-ota-25590-x86_64-2018-10-18.cgz/lkp-cfl-e1/hackbench/0xb8

commit: 
  ba5c5e4a5d ("arm64: move jump_label_init() before parse_early_param()")
  6471384af2 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options")

ba5c5e4a5da443e8 6471384af2a6530696fc0203baf 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     90709            -4.9%      86276        hackbench.throughput
    108209           +42.0%     153632        hackbench.time.involuntary_context_switches
    825777            -3.9%     793179        hackbench.time.minor_page_faults
    712.02            -3.9%     684.38        hackbench.time.user_time
 5.616e+08            -4.3%  5.376e+08        hackbench.workload
      0.84            +0.1        0.95 ±  2%  mpstat.cpu.-1.irq%
      1347 ±  3%     +12.7%       1518 ±  2%  slabinfo.kmalloc-rcl-64.num_objs
      3.49            +4.7%       3.65        turbostat.RAMWatt
    396367            -1.3%     391171        vmstat.system.cs
  14755342            -3.8%   14194624        proc-vmstat.numa_hit
  14755342            -3.8%   14194624        proc-vmstat.numa_local
  14815709            -3.7%   14261066        proc-vmstat.pgalloc_normal
  14779482            -3.8%   14224922        proc-vmstat.pgfree
      8627 ±172%    +682.5%      67514 ± 65%  sched_debug.cfs_rq:/.MIN_vruntime.avg
    137821 ±172%    +585.5%     944775 ± 59%  sched_debug.cfs_rq:/.MIN_vruntime.max
     33372 ±172%    +620.8%     240555 ± 61%  sched_debug.cfs_rq:/.MIN_vruntime.stddev
      8627 ±172%    +682.5%      67514 ± 65%  sched_debug.cfs_rq:/.max_vruntime.avg
    137821 ±172%    +585.5%     944775 ± 59%  sched_debug.cfs_rq:/.max_vruntime.max
     33372 ±172%    +620.8%     240555 ± 61%  sched_debug.cfs_rq:/.max_vruntime.stddev
    556.14 ±  4%      -9.3%     504.21 ±  5%  sched_debug.cpu.clock_task.stddev
      1883 ± 36%     -48.0%     979.75 ± 26%  interrupts.133:IR-PCI-MSI.2097153-edge.eth1-TxRx-0
     46373 ±  4%     -10.1%      41692 ±  4%  interrupts.CPU0.RES:Rescheduling_interrupts
      1883 ± 36%     -48.0%     979.75 ± 26%  interrupts.CPU1.133:IR-PCI-MSI.2097153-edge.eth1-TxRx-0
     45230 ±  5%     -11.7%      39943 ±  4%  interrupts.CPU1.RES:Rescheduling_interrupts
     57716 ±  3%     -12.3%      50601        interrupts.CPU12.RES:Rescheduling_interrupts
     53566 ±  3%     -11.5%      47399        interrupts.CPU13.RES:Rescheduling_interrupts
     52007 ±  4%     -14.0%      44710 ±  5%  interrupts.CPU15.RES:Rescheduling_interrupts
     46887 ±  3%     -12.3%      41140 ±  6%  interrupts.CPU2.RES:Rescheduling_interrupts
     52269 ±  5%     -12.1%      45958 ±  5%  interrupts.CPU3.RES:Rescheduling_interrupts
     57120 ±  7%     -12.0%      50281 ±  4%  interrupts.CPU4.RES:Rescheduling_interrupts
     49253 ±  3%     -11.5%      43586 ±  2%  interrupts.CPU7.RES:Rescheduling_interrupts
    820390           -10.2%     736882        interrupts.RES:Rescheduling_interrupts
     44.93            -2.4%      43.85        perf-stat.i.MPKI
 9.635e+09            -3.5%    9.3e+09        perf-stat.i.branch-instructions
 1.106e+08            -4.2%   1.06e+08        perf-stat.i.branch-misses
     13.44            +0.3       13.72        perf-stat.i.cache-miss-rate%
 2.912e+08            -4.3%  2.786e+08        perf-stat.i.cache-misses
 2.171e+09            -6.3%  2.034e+09        perf-stat.i.cache-references
    397614            -1.4%     391925        perf-stat.i.context-switches
      1.26            +4.2%       1.31        perf-stat.i.cpi
     62650            -3.9%      60222        perf-stat.i.cpu-migrations
    209.19            +4.6%     218.81        perf-stat.i.cycles-between-cache-misses
 1.459e+10            -4.0%  1.401e+10        perf-stat.i.dTLB-loads
 1.025e+10            -4.2%  9.817e+09        perf-stat.i.dTLB-stores
  86435026            -2.5%   84314445        perf-stat.i.iTLB-load-misses
 4.833e+10            -4.0%  4.639e+10        perf-stat.i.instructions
    562.26            -1.5%     553.60        perf-stat.i.instructions-per-iTLB-miss
      0.79            -4.0%       0.76        perf-stat.i.ipc
      0.00 ± 35%      +0.0        0.00 ± 49%  perf-stat.i.node-load-miss-rate%
      0.64 ± 41%     +40.2%       0.90 ± 32%  perf-stat.i.node-load-misses
  21652012            -2.2%   21184023        perf-stat.i.node-loads
      0.57 ±  7%     +76.9%       1.00 ± 33%  perf-stat.i.node-store-misses
  26105714           +79.9%   46970933        perf-stat.i.node-stores
     44.93            -2.4%      43.85        perf-stat.overall.MPKI
     13.41            +0.3       13.69        perf-stat.overall.cache-miss-rate%
      1.26            +4.2%       1.31        perf-stat.overall.cpi
    208.98            +4.6%     218.59        perf-stat.overall.cycles-between-cache-misses
    559.11            -1.6%     550.27        perf-stat.overall.instructions-per-iTLB-miss
      0.79            -4.0%       0.76        perf-stat.overall.ipc
      0.00 ± 42%      +0.0        0.00 ± 33%  perf-stat.overall.node-load-miss-rate%
 9.619e+09            -3.5%  9.285e+09        perf-stat.ps.branch-instructions
 1.104e+08            -4.2%  1.058e+08        perf-stat.ps.branch-misses
 2.907e+08            -4.3%  2.781e+08        perf-stat.ps.cache-misses
 2.168e+09            -6.3%  2.031e+09        perf-stat.ps.cache-references
    396957            -1.4%     391281        perf-stat.ps.context-switches
     62546            -3.9%      60123        perf-stat.ps.cpu-migrations
 1.457e+10            -4.0%  1.399e+10        perf-stat.ps.dTLB-loads
 1.023e+10            -4.2%  9.801e+09        perf-stat.ps.dTLB-stores
  86292583            -2.5%   84176193        perf-stat.ps.iTLB-load-misses
 4.825e+10            -4.0%  4.632e+10        perf-stat.ps.instructions
      0.64 ± 41%     +40.2%       0.90 ± 32%  perf-stat.ps.node-load-misses
  21616330            -2.2%   21149342        perf-stat.ps.node-loads
      0.56 ±  7%     +76.9%       1.00 ± 33%  perf-stat.ps.node-store-misses
  26062702           +79.9%   46894016        perf-stat.ps.node-stores
 2.929e+13            -3.5%  2.827e+13        perf-stat.total.instructions
     39.46            -1.3       38.13        perf-profile.calltrace.cycles-pp.__x64_sys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     39.37            -1.3       38.05        perf-profile.calltrace.cycles-pp.ksys_read.__x64_sys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     38.18            -1.3       36.91        perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.__x64_sys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     35.37            -1.2       34.19        perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.__x64_sys_read.do_syscall_64
     34.98            -1.2       33.81        perf-profile.calltrace.cycles-pp.new_sync_read.__vfs_read.vfs_read.ksys_read.__x64_sys_read
     34.12            -1.1       33.00        perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.__vfs_read.vfs_read.ksys_read
     32.88            -1.0       31.84        perf-profile.calltrace.cycles-pp.sock_recvmsg.sock_read_iter.new_sync_read.__vfs_read.vfs_read
     32.46            -1.0       31.43        perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read.__vfs_read
     32.04            -1.0       31.03        perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read
      2.70 ±  2%      -0.8        1.93        perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      2.16 ±  2%      -0.8        1.41        perf-profile.calltrace.cycles-pp.refcount_add_checked.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
      2.70            -0.7        1.95        perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      2.16 ±  2%      -0.7        1.41        perf-profile.calltrace.cycles-pp.refcount_add_not_zero_checked.refcount_add_checked.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg
     11.94            -0.4       11.50        perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      9.59            -0.4        9.18        perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      9.49            -0.4        9.09        perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      9.20            -0.4        8.85        perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
      8.85            -0.3        8.54        perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
      6.51            -0.3        6.21        perf-profile.calltrace.cycles-pp._copy_to_iter.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      7.29            -0.3        7.00        perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      5.44            -0.2        5.20        perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_to_iter.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
      3.99            -0.2        3.79        perf-profile.calltrace.cycles-pp.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.44 ±  2%      -0.2        2.24 ±  4%  perf-profile.calltrace.cycles-pp.___cache_free.kfree.skb_free_head.skb_release_data.skb_release_all
      3.23            -0.2        3.05        perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      3.28            -0.2        3.10        perf-profile.calltrace.cycles-pp.__audit_syscall_exit.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe
      2.74            -0.2        2.56        perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      3.05            -0.2        2.88        perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
      1.53 ±  3%      -0.2        1.36        perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
      4.44            -0.2        4.27        perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      2.62            -0.2        2.46        perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
      1.95            -0.2        1.79        perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      1.11            -0.1        0.97 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock.___cache_free.kfree.skb_free_head.skb_release_data
      1.53 ±  2%      -0.1        1.39 ±  2%  perf-profile.calltrace.cycles-pp.cache_alloc_refill.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb
      1.02 ±  2%      -0.1        0.89 ±  3%  perf-profile.calltrace.cycles-pp.queued_spin_lock_slowpath._raw_spin_lock.___cache_free.kfree.skb_free_head
      3.69            -0.1        3.56        perf-profile.calltrace.cycles-pp.kmem_cache_free.kfree_skbmem.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      3.83            -0.1        3.70        perf-profile.calltrace.cycles-pp.kfree_skbmem.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      1.60            -0.1        1.47 ±  2%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
      1.85            -0.1        1.73        perf-profile.calltrace.cycles-pp.syscall_trace_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.60            -0.1        1.48 ±  2%  perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      2.70            -0.1        2.59        perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      0.69 ±  3%      -0.1        0.58        perf-profile.calltrace.cycles-pp.queued_spin_lock_slowpath._raw_spin_lock.cache_alloc_refill.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller
      1.77            -0.1        1.66        perf-profile.calltrace.cycles-pp.___cache_free.kmem_cache_free.kfree_skbmem.consume_skb.unix_stream_read_generic
      0.74 ±  2%      -0.1        0.64        perf-profile.calltrace.cycles-pp._raw_spin_lock.cache_alloc_refill.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller.__kmalloc_reserve
      1.82            -0.1        1.72        perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.__x64_sys_write.do_syscall_64
      1.90            -0.1        1.81        perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.98 ±  2%      -0.1        0.89 ±  3%  perf-profile.calltrace.cycles-pp.unroll_tree_refs.__audit_syscall_exit.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.05            -0.1        0.99 ±  3%  perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all
      0.97 ±  2%      -0.1        0.92 ±  2%  perf-profile.calltrace.cycles-pp.__audit_syscall_entry.syscall_trace_enter.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.65            -0.1        0.59 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock.___cache_free.kmem_cache_free.kfree_skbmem.consume_skb
      0.60            -0.1        0.55 ±  3%  perf-profile.calltrace.cycles-pp.queued_spin_lock_slowpath._raw_spin_lock.___cache_free.kmem_cache_free.kfree_skbmem
      0.70            -0.1        0.64        perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      0.86            -0.1        0.81        perf-profile.calltrace.cycles-pp.free_block.___cache_free.kmem_cache_free.kfree_skbmem.consume_skb
      2.03            -0.1        1.98        perf-profile.calltrace.cycles-pp.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      0.66 ±  2%      -0.1        0.60        perf-profile.calltrace.cycles-pp.refcount_inc_not_zero_checked.refcount_inc_checked.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      1.98            -0.0        1.94        perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      0.69 ±  3%      -0.0        0.64 ±  2%  perf-profile.calltrace.cycles-pp.refcount_inc_checked.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      1.94            -0.0        1.89        perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg
      0.75 ±  3%      -0.0        0.71        perf-profile.calltrace.cycles-pp.deactivate_task.__sched_text_start.schedule.schedule_timeout.unix_stream_read_generic
      0.70 ±  2%      -0.0        0.67        perf-profile.calltrace.cycles-pp.cache_alloc_refill.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      1.86 ±  2%      +0.2        2.08        perf-profile.calltrace.cycles-pp.refcount_inc_not_zero_checked.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
     92.65            +0.4       93.01        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     90.21            +0.6       90.79        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +1.0        1.00 ±  4%  perf-profile.calltrace.cycles-pp.memset_erms.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      0.00            +1.1        1.06        perf-profile.calltrace.cycles-pp.memset_erms.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
      4.41            +1.7        6.13        perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
     42.73            +2.2       44.95        perf-profile.calltrace.cycles-pp.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     42.64            +2.2       44.86        perf-profile.calltrace.cycles-pp.ksys_write.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     40.20            +2.4       42.55        perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     37.99            +2.4       40.41        perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.__x64_sys_write.do_syscall_64
     37.73            +2.4       40.18        perf-profile.calltrace.cycles-pp.new_sync_write.__vfs_write.vfs_write.ksys_write.__x64_sys_write
     36.93            +2.5       39.41        perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.__vfs_write.vfs_write.ksys_write
     35.68            +2.5       38.19        perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.__vfs_write.vfs_write
     34.85            +2.5       37.36        perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.__vfs_write
      4.99            +2.8        7.79        perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      5.22            +3.0        8.21        perf-profile.calltrace.cycles-pp.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
     16.62            +4.0       20.60        perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
     12.22            +4.7       16.95        perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
     12.55            +4.7       17.29        perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
     39.76            -1.3       38.41        perf-profile.children.cycles-pp.__x64_sys_read
     39.48            -1.3       38.16        perf-profile.children.cycles-pp.ksys_read
     38.25            -1.3       36.99        perf-profile.children.cycles-pp.vfs_read
     35.48            -1.2       34.28        perf-profile.children.cycles-pp.__vfs_read
     35.06            -1.2       33.89        perf-profile.children.cycles-pp.new_sync_read
     34.37            -1.1       33.23        perf-profile.children.cycles-pp.sock_read_iter
      6.75            -1.1        5.62        perf-profile.children.cycles-pp._raw_spin_lock
     32.74            -1.0       31.70        perf-profile.children.cycles-pp.unix_stream_recvmsg
     33.00            -1.0       31.95        perf-profile.children.cycles-pp.sock_recvmsg
     32.16            -1.0       31.14        perf-profile.children.cycles-pp.unix_stream_read_generic
      2.74            -0.8        1.98        perf-profile.children.cycles-pp.skb_set_owner_w
      2.21 ±  2%      -0.7        1.46        perf-profile.children.cycles-pp.refcount_add_checked
      2.33            -0.7        1.59        perf-profile.children.cycles-pp.refcount_add_not_zero_checked
     12.06            -0.4       11.61        perf-profile.children.cycles-pp.consume_skb
      9.63            -0.4        9.22        perf-profile.children.cycles-pp.unix_stream_read_actor
      9.56            -0.4        9.15        perf-profile.children.cycles-pp.skb_copy_datagram_iter
      9.26            -0.4        8.90        perf-profile.children.cycles-pp.__skb_datagram_iter
      8.99            -0.4        8.63        perf-profile.children.cycles-pp.simple_copy_to_iter
      4.35            -0.4        3.99        perf-profile.children.cycles-pp.queued_spin_lock_slowpath
      7.37            -0.3        7.06        perf-profile.children.cycles-pp.skb_release_all
      4.26 ±  2%      -0.3        3.95        perf-profile.children.cycles-pp.___cache_free
      6.56            -0.3        6.26        perf-profile.children.cycles-pp._copy_to_iter
      7.13            -0.3        6.83        perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      4.06            -0.3        3.79        perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      3.06            -0.3        2.80        perf-profile.children.cycles-pp.syscall_return_via_sysret
      4.12            -0.2        3.91        perf-profile.children.cycles-pp.syscall_slow_exit_work
      2.78            -0.2        2.59        perf-profile.children.cycles-pp.skb_queue_tail
      2.46 ±  2%      -0.2        2.27        perf-profile.children.cycles-pp.__might_sleep
      3.28            -0.2        3.10        perf-profile.children.cycles-pp.skb_release_head_state
      2.74            -0.2        2.57        perf-profile.children.cycles-pp.sock_wfree
      3.31            -0.2        3.14 ±  2%  perf-profile.children.cycles-pp.unix_destruct_scm
      4.49            -0.2        4.32        perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
      3.48            -0.2        3.31        perf-profile.children.cycles-pp.__audit_syscall_exit
      2.24            -0.2        2.08        perf-profile.children.cycles-pp.cache_alloc_refill
      3.79            -0.2        3.64        perf-profile.children.cycles-pp.kmem_cache_free
      3.90            -0.1        3.77        perf-profile.children.cycles-pp.kfree_skbmem
      1.62            -0.1        1.49        perf-profile.children.cycles-pp.entry_SYSCALL_64
      2.77            -0.1        2.65        perf-profile.children.cycles-pp._copy_from_iter
      1.94            -0.1        1.82        perf-profile.children.cycles-pp.syscall_trace_enter
      2.52            -0.1        2.40        perf-profile.children.cycles-pp.__fget_light
      3.14 ±  2%      -0.1        3.02        perf-profile.children.cycles-pp.__check_object_size
      1.58 ±  3%      -0.1        1.46        perf-profile.children.cycles-pp.___might_sleep
      1.61            -0.1        1.50 ±  2%  perf-profile.children.cycles-pp.skb_unlink
      2.67            -0.1        2.56        perf-profile.children.cycles-pp.__fdget_pos
      1.77 ±  2%      -0.1        1.67        perf-profile.children.cycles-pp.free_block
      1.10 ±  3%      -0.1        1.01 ±  3%  perf-profile.children.cycles-pp.__audit_syscall_entry
      0.57 ±  5%      -0.1        0.48 ±  9%  perf-profile.children.cycles-pp.memcg_kmem_put_cache
      1.08 ±  2%      -0.1        1.00 ±  2%  perf-profile.children.cycles-pp.unroll_tree_refs
      0.93 ±  4%      -0.1        0.84        perf-profile.children.cycles-pp.__might_fault
      0.18 ±  7%      -0.1        0.11 ±  4%  perf-profile.children.cycles-pp.should_failslab
      0.56 ±  6%      -0.1        0.50        perf-profile.children.cycles-pp.rcu_all_qs
      2.70            -0.1        2.64        perf-profile.children.cycles-pp.schedule
      2.75            -0.1        2.69        perf-profile.children.cycles-pp.schedule_timeout
      2.83            -0.1        2.78        perf-profile.children.cycles-pp.__sched_text_start
      1.07            -0.1        1.02        perf-profile.children.cycles-pp.__list_del_entry_valid
      0.70 ±  3%      -0.1        0.65 ±  2%  perf-profile.children.cycles-pp.refcount_inc_checked
      0.30            -0.0        0.25 ±  6%  perf-profile.children.cycles-pp.copyin
      0.67            -0.0        0.62        perf-profile.children.cycles-pp._cond_resched
      1.19            -0.0        1.15 ±  2%  perf-profile.children.cycles-pp.security_file_permission
      0.74 ±  2%      -0.0        0.70 ±  2%  perf-profile.children.cycles-pp.wait_for_unix_gc
      1.14            -0.0        1.10 ±  2%  perf-profile.children.cycles-pp.unix_write_space
      0.32 ±  2%      -0.0        0.29 ±  3%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.98            -0.0        0.94        perf-profile.children.cycles-pp.deactivate_task
      0.22            -0.0        0.20 ±  5%  perf-profile.children.cycles-pp.update_rq_clock
      0.73            -0.0        0.71 ±  2%  perf-profile.children.cycles-pp.dequeue_task_fair
      0.16 ±  2%      -0.0        0.15        perf-profile.children.cycles-pp.set_next_entity
      0.08 ±  8%      +0.0        0.10 ±  4%  perf-profile.children.cycles-pp.cache_grow_begin
      0.09 ±  4%      +0.0        0.12 ± 11%  perf-profile.children.cycles-pp.maybe_add_creds
      0.00            +0.1        0.08 ±  6%  perf-profile.children.cycles-pp.memset
      0.97 ±  5%      +0.2        1.13 ±  3%  perf-profile.children.cycles-pp.__virt_addr_valid
      2.52 ±  2%      +0.2        2.69        perf-profile.children.cycles-pp.refcount_inc_not_zero_checked
     92.69            +0.4       93.05        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     90.39            +0.6       90.97        perf-profile.children.cycles-pp.do_syscall_64
      4.64            +1.8        6.49        perf-profile.children.cycles-pp.kmem_cache_alloc_node_trace
      0.00            +2.1        2.11 ±  2%  perf-profile.children.cycles-pp.memset_erms
     43.03            +2.2       45.23        perf-profile.children.cycles-pp.__x64_sys_write
     42.76            +2.2       44.96        perf-profile.children.cycles-pp.ksys_write
     40.25            +2.3       42.59        perf-profile.children.cycles-pp.vfs_write
     38.06            +2.4       40.48        perf-profile.children.cycles-pp.__vfs_write
     37.80            +2.4       40.24        perf-profile.children.cycles-pp.new_sync_write
     37.11            +2.5       39.57        perf-profile.children.cycles-pp.sock_write_iter
     35.75            +2.5       38.24        perf-profile.children.cycles-pp.sock_sendmsg
     35.18            +2.5       37.70        perf-profile.children.cycles-pp.unix_stream_sendmsg
      5.05            +2.8        7.89        perf-profile.children.cycles-pp.__kmalloc_node_track_caller
      5.26            +3.0        8.30        perf-profile.children.cycles-pp.__kmalloc_reserve
     16.66            +4.0       20.62        perf-profile.children.cycles-pp.sock_alloc_send_pskb
     12.30            +4.7       17.02        perf-profile.children.cycles-pp.__alloc_skb
     12.59            +4.7       17.33        perf-profile.children.cycles-pp.alloc_skb_with_frags
      3.43 ±  2%      -0.8        2.61        perf-profile.self.cycles-pp._raw_spin_lock
      2.31            -0.7        1.57        perf-profile.self.cycles-pp.refcount_add_not_zero_checked
      4.33            -0.3        3.98        perf-profile.self.cycles-pp.queued_spin_lock_slowpath
      7.10            -0.3        6.80        perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      3.03            -0.3        2.77        perf-profile.self.cycles-pp.syscall_return_via_sysret
      1.87            -0.2        1.62 ±  2%  perf-profile.self.cycles-pp.__check_object_size
      3.01            -0.2        2.80        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      1.76            -0.2        1.59        perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.37 ±  2%      -0.1        1.24 ±  2%  perf-profile.self.cycles-pp.sock_def_readable
      1.62            -0.1        1.49        perf-profile.self.cycles-pp.entry_SYSCALL_64
      2.46            -0.1        2.34        perf-profile.self.cycles-pp.__fget_light
      1.80            -0.1        1.69        perf-profile.self.cycles-pp.unix_stream_read_generic
      1.49 ±  2%      -0.1        1.38 ±  2%  perf-profile.self.cycles-pp.___might_sleep
      0.90 ±  3%      -0.1        0.81 ±  2%  perf-profile.self.cycles-pp.__audit_syscall_entry
      1.07 ±  2%      -0.1        0.98 ±  3%  perf-profile.self.cycles-pp.unroll_tree_refs
      1.14 ±  2%      -0.1        1.06        perf-profile.self.cycles-pp.sock_read_iter
      0.66 ±  3%      -0.1        0.58 ±  2%  perf-profile.self.cycles-pp.sock_wfree
      1.04 ±  3%      -0.1        0.98        perf-profile.self.cycles-pp.__might_sleep
      1.04            -0.1        0.98        perf-profile.self.cycles-pp.__list_del_entry_valid
      1.86            -0.1        1.80        perf-profile.self.cycles-pp.__audit_syscall_exit
      0.46 ±  8%      -0.1        0.40 ±  3%  perf-profile.self.cycles-pp.rcu_all_qs
      0.98 ±  2%      -0.1        0.93 ±  4%  perf-profile.self.cycles-pp.free_block
      0.74 ±  2%      -0.1        0.69 ±  2%  perf-profile.self.cycles-pp.vfs_read
      0.58            -0.0        0.54 ±  2%  perf-profile.self.cycles-pp.syscall_slow_exit_work
      0.11 ±  9%      -0.0        0.07 ±  5%  perf-profile.self.cycles-pp.should_failslab
      0.74            -0.0        0.70        perf-profile.self.cycles-pp.new_sync_read
      0.37 ±  3%      -0.0        0.34 ±  2%  perf-profile.self.cycles-pp._cond_resched
      0.64            -0.0        0.60        perf-profile.self.cycles-pp._copy_from_iter
      0.18 ±  2%      -0.0        0.15 ±  5%  perf-profile.self.cycles-pp.copyin
      0.30 ±  5%      -0.0        0.27 ±  5%  perf-profile.self.cycles-pp.__skb_datagram_iter
      0.25 ±  4%      -0.0        0.23 ±  6%  perf-profile.self.cycles-pp.simple_copy_to_iter
      0.23            -0.0        0.21 ±  5%  perf-profile.self.cycles-pp.sock_sendmsg
      0.14 ±  5%      -0.0        0.12        perf-profile.self.cycles-pp.__list_add_valid
      0.18            -0.0        0.16 ±  4%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.12 ±  4%      -0.0        0.11 ±  6%  perf-profile.self.cycles-pp.security_socket_sendmsg
      0.09 ±  4%      +0.0        0.11 ±  9%  perf-profile.self.cycles-pp.maybe_add_creds
      0.11 ± 21%      +0.0        0.15 ± 24%  perf-profile.self.cycles-pp.kmalloc_slab
      0.19 ±  5%      +0.1        0.26 ± 12%  perf-profile.self.cycles-pp.__kmalloc_reserve
      1.11 ±  2%      +0.1        1.19        perf-profile.self.cycles-pp.kfree
      0.13 ± 11%      +0.1        0.22 ±  8%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      0.94 ±  5%      +0.1        1.09 ±  3%  perf-profile.self.cycles-pp.__virt_addr_valid
      2.51 ±  2%      +0.2        2.68        perf-profile.self.cycles-pp.refcount_inc_not_zero_checked
      1.66 ±  5%      +0.8        2.42        perf-profile.self.cycles-pp.__alloc_skb
      1.26            +1.9        3.15 ±  2%  perf-profile.self.cycles-pp.kmem_cache_alloc_node_trace
      0.00            +2.1        2.07 ±  2%  perf-profile.self.cycles-pp.memset_erms


                                                                                
                                hackbench.throughput                            
                                                                                
  100000 +-+----------------------------------------------------------------+   
   90000 +-+++  +.++  +.++.++.+  +  ++.+  ++  +.++  +.++  +.+  ++  +.+  +  +|   
         |O OO OO O:  O O   O O  OO :O O  OO  O O: OO  O O: OO :O O: O  OO :|   
   80000 +-+ :  :  :  :       :  :  :  :  ::  :  :  :  :  : :  ::  : :  :  :|   
   70000 +-+ :  :  :  :       :  :  :  :  ::  :  :  :  :  : :  ::  : :  :  :|   
         |:   : :   : :       : :: :   : :  : :   : :   : : : :  : : : :: : |   
   60000 +-+  : :   : :       : :: :   : :  : :   : :   : : : :  : : : :: : |   
   50000 +-+  : :   : :       : :: :   : :  : :   : :   : : : :  : : : :: : |   
   40000 +-+  ::    ::         :: ::    ::  ::    ::    ::   ::  ::   :: :: |   
         |    ::    ::         :: ::    ::  ::    ::    ::   ::  ::   :: :: |   
   30000 +-+  ::    ::         :: ::    ::  ::    ::    ::   ::  ::   :: :: |   
   20000 +-+   :     :         :  :     :    :     :     :   :    :   :  :  |   
         |     :     :         :  :     :    :     :     :   :    :   :  :  |   
   10000 +-+   :     :         :  :     :    :     :     :   :    :   :  :  |   
       0 O-+-------O-O---O-O---O----O---O----O---O----O---O----O---O--O-----+   
                                                                                
                                                                                                                                                                
                     hackbench.time.involuntary_context_switches                
                                                                                
  180000 +-+----------------------------------------------------------------+   
         |                             O  O     O  OO  O                    |   
  160000 +O+ O  O O   O O   O O  OO  O     O  O          O      O O  O  OO  |   
  140000 +-+O  O                                            OO              |   
         |                                             +                    |   
  120000 +-+       +  +.++.++.+  +                     :  +.   +            |   
  100000 +-+++  +.+:  :       :  :  ++.+  ++  +.++  +.+:  : +  :+  +.+  +  +|   
         |:  :  :  :  :       :  :  :  :  ::  :  :  :   : : :  ::  : :  :  :|   
   80000 +-+  : :   : :       : ::  :  :  : : :   : :   : : : :  : : :  :  :|   
   60000 +-+  : :   : :       : :: :   : :  : :   : :   : : : :  : : : :: : |   
         |:   : :   ::         :: ::    ::  : :   : :   ::   ::  : :  :: :: |   
   40000 +-+  ::    ::         :: ::    ::  ::    ::    ::   ::  ::   :: :: |   
   20000 +-+   :     :         :  ::    ::   :     :     :   :    :   :: :: |   
         |     :     :         :  :     :    :     :     :   :    :   :  :  |   
       0 O-+-------O-O---O-O---O----O---O----O---O----O---O----O---O--O-----+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
  gcc-7/performance/x86_64-rhel-7.2-clear_lck_7595/process/100%/clear-ota-25590-x86_64-2018-10-18.cgz/lkp-cfl-e1/hackbench/0xb8

commit: 
  ba5c5e4a5d ("arm64: move jump_label_init() before parse_early_param()")
  6471384af2 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options")

ba5c5e4a5da443e8 6471384af2a6530696fc0203baf 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     88494            -4.3%      84657        hackbench.throughput
    138438 ±  4%     +31.9%     182656 ±  7%  hackbench.time.involuntary_context_switches
    694.79            -5.9%     653.85        hackbench.time.user_time
 5.568e+08            -4.3%  5.328e+08        hackbench.workload
   8544005 ± 38%     -45.2%    4685694 ±  6%  cpuidle.C1E.time
      5.31 ± 11%     +29.2%       6.86 ± 16%  sched_debug.cpu.nr_uninterruptible.stddev
      3.48            +4.6%       3.64        turbostat.RAMWatt
      0.84 ±  2%      +0.1        0.95        mpstat.cpu.-1.irq%
      0.05 ±  3%      +0.0        0.06 ± 10%  mpstat.cpu.-1.soft%
  15632269            -4.1%   14984464 ±  3%  proc-vmstat.numa_hit
  15632269            -4.1%   14984464 ±  3%  proc-vmstat.numa_local
  15715101            -4.2%   15056704 ±  3%  proc-vmstat.pgalloc_normal
  15686075            -4.2%   15022294 ±  3%  proc-vmstat.pgfree
     45.36            -2.9%      44.03        perf-stat.i.MPKI
 9.663e+09            -2.7%  9.402e+09        perf-stat.i.branch-instructions
 1.088e+08            -2.7%  1.058e+08        perf-stat.i.branch-misses
     13.34            +0.3       13.62        perf-stat.i.cache-miss-rate%
 2.908e+08            -4.1%  2.787e+08        perf-stat.i.cache-misses
 2.179e+09            -6.0%  2.048e+09        perf-stat.i.cache-references
      1.26            +3.4%       1.31        perf-stat.i.cpi
     61720            -2.9%      59924        perf-stat.i.cpu-migrations
    210.15            +4.0%     218.46        perf-stat.i.cycles-between-cache-misses
 1.449e+10            -3.3%  1.401e+10        perf-stat.i.dTLB-loads
 1.015e+10            -3.5%   9.79e+09        perf-stat.i.dTLB-stores
 4.808e+10            -3.3%  4.651e+10        perf-stat.i.instructions
      0.79            -3.3%       0.77        perf-stat.i.ipc
  20904568            -2.2%   20436868        perf-stat.i.node-loads
  25841390           +79.8%   46459814        perf-stat.i.node-stores
     45.31            -2.8%      44.03        perf-stat.overall.MPKI
     13.35            +0.3       13.61        perf-stat.overall.cache-miss-rate%
      1.26            +3.5%       1.31        perf-stat.overall.cpi
    208.67            +4.4%     217.94        perf-stat.overall.cycles-between-cache-misses
      0.79            -3.4%       0.77        perf-stat.overall.ipc
 9.647e+09            -2.7%  9.387e+09        perf-stat.ps.branch-instructions
 1.086e+08            -2.7%  1.056e+08        perf-stat.ps.branch-misses
 2.903e+08            -4.1%  2.783e+08        perf-stat.ps.cache-misses
 2.175e+09            -6.0%  2.045e+09        perf-stat.ps.cache-references
     61620            -2.9%      59827        perf-stat.ps.cpu-migrations
 1.447e+10            -3.3%  1.399e+10        perf-stat.ps.dTLB-loads
 1.013e+10            -3.5%  9.774e+09        perf-stat.ps.dTLB-stores
   4.8e+10            -3.3%  4.643e+10        perf-stat.ps.instructions
  20870674            -2.2%   20403728        perf-stat.ps.node-loads
  25799504           +79.8%   46384429        perf-stat.ps.node-stores
 2.964e+13            -3.4%  2.863e+13        perf-stat.total.instructions
     38.02            -1.3       36.70        perf-profile.calltrace.cycles-pp.ksys_read.__x64_sys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     38.11            -1.3       36.80        perf-profile.calltrace.cycles-pp.__x64_sys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     36.86            -1.3       35.58        perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.__x64_sys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     34.13            -1.2       32.97        perf-profile.calltrace.cycles-pp.__vfs_read.vfs_read.ksys_read.__x64_sys_read.do_syscall_64
     33.76            -1.1       32.61        perf-profile.calltrace.cycles-pp.new_sync_read.__vfs_read.vfs_read.ksys_read.__x64_sys_read
     32.94            -1.1       31.84        perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.__vfs_read.vfs_read.ksys_read
     31.73            -1.0       30.70        perf-profile.calltrace.cycles-pp.sock_recvmsg.sock_read_iter.new_sync_read.__vfs_read.vfs_read
     31.32            -1.0       30.30        perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read.__vfs_read
     30.92            -1.0       29.92        perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read
      2.61 ±  3%      -0.6        1.97        perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      2.02 ±  4%      -0.6        1.39        perf-profile.calltrace.cycles-pp.refcount_add_not_zero_checked.refcount_add_checked.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg
      2.03 ±  4%      -0.6        1.39        perf-profile.calltrace.cycles-pp.refcount_add_checked.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
      2.68            -0.6        2.05 ±  2%  perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      6.64 ±  3%      -0.4        6.25 ±  2%  perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      9.06            -0.4        8.68 ±  2%  perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      8.98            -0.4        8.59 ±  2%  perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      8.74            -0.4        8.36 ±  2%  perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
      8.41            -0.4        8.05 ±  2%  perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
     11.82            -0.3       11.47        perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      6.11            -0.3        5.79        perf-profile.calltrace.cycles-pp._copy_to_iter.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
      5.04            -0.3        4.79 ±  2%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_to_iter.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
      3.55 ±  3%      -0.2        3.36 ±  2%  perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.__wake_up_sync_key.sock_def_readable.unix_stream_sendmsg
      3.57            -0.2        3.38        perf-profile.calltrace.cycles-pp.kmem_cache_free.kfree_skbmem.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      7.34            -0.2        7.16        perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      3.67            -0.2        3.50        perf-profile.calltrace.cycles-pp.kfree_skbmem.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      3.47            -0.2        3.31        perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
      2.90            -0.2        2.75        perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
      1.85 ±  3%      -0.1        1.70 ±  4%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.__wake_up_sync_key.sock_def_readable.unix_stream_sendmsg
      1.53 ±  3%      -0.1        1.39 ±  4%  perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
      2.32 ±  2%      -0.1        2.18        perf-profile.calltrace.cycles-pp.___cache_free.kfree.skb_free_head.skb_release_data.skb_release_all
      4.39            -0.1        4.25        perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      1.77 ±  3%      -0.1        1.63 ±  4%  perf-profile.calltrace.cycles-pp.queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.__wake_up_sync_key.sock_def_readable
      3.28            -0.1        3.15        perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
      2.04 ±  2%      -0.1        1.92 ±  2%  perf-profile.calltrace.cycles-pp.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      2.67            -0.1        2.55        perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      1.96 ±  2%      -0.1        1.84 ±  2%  perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg
      2.00 ±  2%      -0.1        1.88 ±  2%  perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      2.68 ±  2%      -0.1        2.57 ±  2%  perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      1.43 ±  2%      -0.1        1.33        perf-profile.calltrace.cycles-pp.cache_alloc_refill.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb
      1.01 ±  3%      -0.1        0.91 ±  2%  perf-profile.calltrace.cycles-pp._raw_spin_lock.___cache_free.kfree.skb_free_head.skb_release_data
      3.91            -0.1        3.81        perf-profile.calltrace.cycles-pp.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.40 ±  4%      -0.1        1.29 ±  2%  perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      1.71 ±  2%      -0.1        1.61        perf-profile.calltrace.cycles-pp.___cache_free.kmem_cache_free.kfree_skbmem.consume_skb.unix_stream_read_generic
      0.93 ±  3%      -0.1        0.83 ±  2%  perf-profile.calltrace.cycles-pp.queued_spin_lock_slowpath._raw_spin_lock.___cache_free.kfree.skb_free_head
      3.24            -0.1        3.15        perf-profile.calltrace.cycles-pp.__audit_syscall_exit.syscall_slow_exit_work.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.38 ±  2%      -0.1        1.29        perf-profile.calltrace.cycles-pp.rw_verify_area.vfs_read.ksys_read.__x64_sys_read.do_syscall_64
      3.74            -0.1        3.66        perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      0.96 ±  4%      -0.1        0.89 ±  2%  perf-profile.calltrace.cycles-pp.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task
      0.67 ±  3%      -0.1        0.60        perf-profile.calltrace.cycles-pp._raw_spin_lock.cache_alloc_refill.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller.__kmalloc_reserve
      0.97            -0.1        0.90        perf-profile.calltrace.cycles-pp.security_file_permission.rw_verify_area.vfs_read.ksys_read.__x64_sys_read
      0.61 ±  3%      -0.1        0.55        perf-profile.calltrace.cycles-pp.queued_spin_lock_slowpath._raw_spin_lock.cache_alloc_refill.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller
      0.81 ±  4%      -0.1        0.75        perf-profile.calltrace.cycles-pp.arch_stack_walk.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
      0.57 ±  5%      -0.0        0.52 ±  2%  perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
      0.61 ±  2%      -0.0        0.57 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
      0.66            -0.0        0.61 ±  3%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      0.65            -0.0        0.61 ±  4%  perf-profile.calltrace.cycles-pp.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
      1.13 ±  2%      +0.1        1.23        perf-profile.calltrace.cycles-pp.ksize.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      1.10            +0.1        1.21        perf-profile.calltrace.cycles-pp.__ksize.ksize.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
      0.38 ± 57%      +0.1        0.53        perf-profile.calltrace.cycles-pp.refcount_dec_and_test_checked.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg
      1.80 ±  2%      +0.2        1.96        perf-profile.calltrace.cycles-pp.refcount_inc_not_zero_checked.unix_stream_read_generic.unix_stream_recvmsg.sock_recvmsg.sock_read_iter
     92.56            +0.4       92.96        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
     90.14            +0.7       90.81        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.00            +0.9        0.86 ±  6%  perf-profile.calltrace.cycles-pp.memset_erms.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
      0.00            +0.9        0.89 ±  8%  perf-profile.calltrace.cycles-pp.memset_erms.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
      4.35            +1.5        5.85        perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node_trace.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
     44.05            +2.2       46.24        perf-profile.calltrace.cycles-pp.ksys_write.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     44.14            +2.2       46.34        perf-profile.calltrace.cycles-pp.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     41.66            +2.3       43.94        perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     39.46            +2.4       41.82        perf-profile.calltrace.cycles-pp.__vfs_write.vfs_write.ksys_write.__x64_sys_write.do_syscall_64
     39.21            +2.4       41.58        perf-profile.calltrace.cycles-pp.new_sync_write.__vfs_write.vfs_write.ksys_write.__x64_sys_write
     38.46            +2.4       40.85        perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.__vfs_write.vfs_write.ksys_write
      4.91            +2.4        7.30        perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
     37.20            +2.4       39.63        perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.__vfs_write.vfs_write
     36.36            +2.4       38.80        perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.__vfs_write
      5.15            +2.6        7.74        perf-profile.calltrace.cycles-pp.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
     16.15            +3.9       20.05        perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
     11.73            +4.5       16.21        perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
     12.06            +4.5       16.55        perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
     38.39            -1.3       37.06        perf-profile.children.cycles-pp.__x64_sys_read
     38.13            -1.3       36.81        perf-profile.children.cycles-pp.ksys_read
     36.94            -1.3       35.66        perf-profile.children.cycles-pp.vfs_read
     34.24            -1.2       33.07        perf-profile.children.cycles-pp.__vfs_read
     33.84            -1.2       32.68        perf-profile.children.cycles-pp.new_sync_read
     33.17            -1.1       32.06        perf-profile.children.cycles-pp.sock_read_iter
     31.60            -1.0       30.55        perf-profile.children.cycles-pp.unix_stream_recvmsg
     31.85            -1.0       30.81        perf-profile.children.cycles-pp.sock_recvmsg
     31.02            -1.0       30.02        perf-profile.children.cycles-pp.unix_stream_read_generic
      6.85            -1.0        5.88        perf-profile.children.cycles-pp._raw_spin_lock
      2.65 ±  3%      -0.6        2.00        perf-profile.children.cycles-pp.skb_set_owner_w
      2.08 ±  4%      -0.6        1.44        perf-profile.children.cycles-pp.refcount_add_checked
      2.23 ±  4%      -0.6        1.62        perf-profile.children.cycles-pp.refcount_add_not_zero_checked
      6.85 ±  2%      -0.4        6.44 ±  2%  perf-profile.children.cycles-pp.sock_def_readable
      9.10            -0.4        8.70 ±  2%  perf-profile.children.cycles-pp.unix_stream_read_actor
      9.04            -0.4        8.65 ±  2%  perf-profile.children.cycles-pp.skb_copy_datagram_iter
      8.79            -0.4        8.41 ±  2%  perf-profile.children.cycles-pp.__skb_datagram_iter
      8.50            -0.4        8.14 ±  2%  perf-profile.children.cycles-pp.simple_copy_to_iter
     11.93            -0.3       11.59        perf-profile.children.cycles-pp.consume_skb
      5.62            -0.3        5.29        perf-profile.children.cycles-pp.queued_spin_lock_slowpath
      6.17            -0.3        5.84        perf-profile.children.cycles-pp._copy_to_iter
      6.71            -0.3        6.42        perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      4.08            -0.2        3.84        perf-profile.children.cycles-pp.___cache_free
      3.02            -0.2        2.80 ±  2%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      5.15            -0.2        4.93        perf-profile.children.cycles-pp._raw_spin_lock_irqsave
      3.66            -0.2        3.46        perf-profile.children.cycles-pp.kmem_cache_free
      7.41            -0.2        7.22        perf-profile.children.cycles-pp.skb_release_all
      3.74            -0.2        3.58        perf-profile.children.cycles-pp.kfree_skbmem
      3.01            -0.2        2.85        perf-profile.children.cycles-pp.sock_wfree
      3.51            -0.2        3.36        perf-profile.children.cycles-pp.skb_release_head_state
      2.38 ±  2%      -0.1        2.23        perf-profile.children.cycles-pp.__might_sleep
      4.44            -0.1        4.30        perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
      3.53            -0.1        3.40        perf-profile.children.cycles-pp.unix_destruct_scm
      1.55 ±  3%      -0.1        1.42 ±  2%  perf-profile.children.cycles-pp.___might_sleep
      2.10 ±  2%      -0.1        1.97        perf-profile.children.cycles-pp.cache_alloc_refill
      2.70            -0.1        2.58        perf-profile.children.cycles-pp.skb_queue_tail
      2.75 ±  2%      -0.1        2.64 ±  2%  perf-profile.children.cycles-pp._copy_from_iter
      1.84            -0.1        1.74        perf-profile.children.cycles-pp.rw_verify_area
      4.03            -0.1        3.93        perf-profile.children.cycles-pp.syscall_slow_exit_work
      1.06 ±  4%      -0.1        0.96 ±  3%  perf-profile.children.cycles-pp.unwind_next_frame
      1.41 ±  4%      -0.1        1.31 ±  2%  perf-profile.children.cycles-pp.skb_unlink
      2.71            -0.1        2.61        perf-profile.children.cycles-pp.schedule_timeout
      2.83            -0.1        2.73        perf-profile.children.cycles-pp.__sched_text_start
      3.43            -0.1        3.34        perf-profile.children.cycles-pp.__audit_syscall_exit
      2.65            -0.1        2.57        perf-profile.children.cycles-pp.schedule
      1.22 ±  4%      -0.1        1.14        perf-profile.children.cycles-pp.stack_trace_save_tsk
      1.15            -0.1        1.07        perf-profile.children.cycles-pp.security_file_permission
      0.58 ±  5%      -0.1        0.50 ±  3%  perf-profile.children.cycles-pp.memcg_kmem_put_cache
      2.65            -0.1        2.58 ±  2%  perf-profile.children.cycles-pp.__fdget_pos
      1.06 ±  3%      -0.1        0.99        perf-profile.children.cycles-pp.arch_stack_walk
      0.90 ±  3%      -0.1        0.83 ±  3%  perf-profile.children.cycles-pp.__might_fault
      1.42 ±  2%      -0.1        1.37        perf-profile.children.cycles-pp.__fsnotify_parent
      0.17 ±  7%      -0.1        0.12 ± 10%  perf-profile.children.cycles-pp.should_failslab
      1.03            -0.1        0.98        perf-profile.children.cycles-pp.__list_del_entry_valid
      0.89 ±  2%      -0.0        0.85 ±  2%  perf-profile.children.cycles-pp.deactivate_task
      0.30            -0.0        0.26 ±  4%  perf-profile.children.cycles-pp.copyin
      1.07 ±  2%      -0.0        1.04        perf-profile.children.cycles-pp.unroll_tree_refs
      1.01            -0.0        0.98        perf-profile.children.cycles-pp.mutex_lock
      0.35 ±  2%      -0.0        0.32 ±  5%  perf-profile.children.cycles-pp.dequeue_entity
      0.36 ±  3%      -0.0        0.34 ±  2%  perf-profile.children.cycles-pp.select_idle_sibling
      0.25 ±  2%      -0.0        0.23 ±  2%  perf-profile.children.cycles-pp.copyout
      0.33 ±  3%      -0.0        0.31 ±  3%  perf-profile.children.cycles-pp.path_put
      0.56            +0.0        0.59 ±  2%  perf-profile.children.cycles-pp.refcount_dec_and_test_checked
      0.08 ± 10%      +0.0        0.11 ±  7%  perf-profile.children.cycles-pp.cache_grow_begin
      0.11 ±  6%      +0.0        0.15 ± 14%  perf-profile.children.cycles-pp.kmalloc_slab
      0.00            +0.1        0.08 ± 10%  perf-profile.children.cycles-pp.memset
      1.15            +0.1        1.26        perf-profile.children.cycles-pp.__ksize
      1.18 ±  2%      +0.1        1.30        perf-profile.children.cycles-pp.ksize
      2.52 ±  2%      +0.2        2.69        perf-profile.children.cycles-pp.refcount_inc_not_zero_checked
     92.60            +0.4       93.00        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     90.32            +0.7       90.98        perf-profile.children.cycles-pp.do_syscall_64
      4.57            +1.6        6.21        perf-profile.children.cycles-pp.kmem_cache_alloc_node_trace
      0.00            +1.8        1.77 ±  8%  perf-profile.children.cycles-pp.memset_erms
     44.45            +2.2       46.62        perf-profile.children.cycles-pp.__x64_sys_write
     44.16            +2.2       46.36        perf-profile.children.cycles-pp.ksys_write
     41.70            +2.3       43.98        perf-profile.children.cycles-pp.vfs_write
     39.54            +2.4       41.89        perf-profile.children.cycles-pp.__vfs_write
     39.29            +2.4       41.66        perf-profile.children.cycles-pp.new_sync_write
     38.63            +2.4       41.02        perf-profile.children.cycles-pp.sock_write_iter
      4.98            +2.4        7.41        perf-profile.children.cycles-pp.__kmalloc_node_track_caller
     37.26            +2.4       39.70        perf-profile.children.cycles-pp.sock_sendmsg
     36.68            +2.4       39.13        perf-profile.children.cycles-pp.unix_stream_sendmsg
      5.18            +2.6        7.80        perf-profile.children.cycles-pp.__kmalloc_reserve
     16.19            +3.9       20.09        perf-profile.children.cycles-pp.sock_alloc_send_pskb
     11.81            +4.5       16.28        perf-profile.children.cycles-pp.__alloc_skb
     12.11            +4.5       16.59        perf-profile.children.cycles-pp.alloc_skb_with_frags
      3.37            -0.7        2.63        perf-profile.self.cycles-pp._raw_spin_lock
      2.21 ±  3%      -0.6        1.60        perf-profile.self.cycles-pp.refcount_add_not_zero_checked
      5.60            -0.3        5.27        perf-profile.self.cycles-pp.queued_spin_lock_slowpath
      6.68            -0.3        6.39        perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      1.77            -0.2        1.55        perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      2.98            -0.2        2.77 ±  2%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      1.83 ±  4%      -0.2        1.65 ±  6%  perf-profile.self.cycles-pp.__check_object_size
      1.45 ±  3%      -0.1        1.34 ±  2%  perf-profile.self.cycles-pp.___might_sleep
      2.98            -0.1        2.88        perf-profile.self.cycles-pp._raw_spin_lock_irqsave
      0.65            -0.1        0.55 ±  2%  perf-profile.self.cycles-pp.sock_wfree
      1.71            -0.1        1.61 ±  2%  perf-profile.self.cycles-pp.unix_stream_read_generic
      2.94            -0.1        2.85        perf-profile.self.cycles-pp.unix_stream_sendmsg
      1.85            -0.1        1.78        perf-profile.self.cycles-pp.kmem_cache_free
      0.92 ±  3%      -0.1        0.85 ±  4%  perf-profile.self.cycles-pp.__audit_syscall_entry
      1.00 ±  2%      -0.1        0.94        perf-profile.self.cycles-pp.__list_del_entry_valid
      1.10 ±  3%      -0.1        1.05 ±  2%  perf-profile.self.cycles-pp.sock_read_iter
      0.50 ±  4%      -0.1        0.45 ±  4%  perf-profile.self.cycles-pp.memcg_kmem_put_cache
      0.71 ±  3%      -0.1        0.66        perf-profile.self.cycles-pp.new_sync_read
      1.00 ±  2%      -0.0        0.95 ±  3%  perf-profile.self.cycles-pp.__might_sleep
      1.83            -0.0        1.79        perf-profile.self.cycles-pp.__audit_syscall_exit
      0.34 ±  5%      -0.0        0.30 ±  4%  perf-profile.self.cycles-pp.skb_copy_datagram_from_iter
      0.18 ±  4%      -0.0        0.14 ±  5%  perf-profile.self.cycles-pp.copyin
      0.40 ±  2%      -0.0        0.37 ±  5%  perf-profile.self.cycles-pp.skb_set_owner_w
      0.10 ±  8%      -0.0        0.08 ±  5%  perf-profile.self.cycles-pp.should_failslab
      0.23 ±  3%      -0.0        0.21 ±  3%  perf-profile.self.cycles-pp.skb_copy_datagram_iter
      0.13 ±  6%      -0.0        0.11 ±  7%  perf-profile.self.cycles-pp.update_rq_clock
      0.14 ±  6%      -0.0        0.12 ±  3%  perf-profile.self.cycles-pp.stack_trace_consume_entry_nosched
      0.11 ±  4%      -0.0        0.10 ±  4%  perf-profile.self.cycles-pp.__update_load_avg_se
      0.11            +0.0        0.12 ±  4%  perf-profile.self.cycles-pp.kfree_skbmem
      0.31 ±  2%      +0.0        0.34 ±  2%  perf-profile.self.cycles-pp.skb_release_data
      0.10 ± 11%      +0.0        0.13 ± 14%  perf-profile.self.cycles-pp.kmalloc_slab
      0.18 ±  2%      +0.0        0.23 ±  4%  perf-profile.self.cycles-pp.__kmalloc_reserve
      0.15 ±  3%      +0.1        0.21 ±  2%  perf-profile.self.cycles-pp.__kmalloc_node_track_caller
      1.07 ±  2%      +0.1        1.18 ±  2%  perf-profile.self.cycles-pp.kfree
      1.12            +0.1        1.24        perf-profile.self.cycles-pp.__ksize
      2.51 ±  2%      +0.2        2.67        perf-profile.self.cycles-pp.refcount_inc_not_zero_checked
      1.38 ± 10%      +1.0        2.39        perf-profile.self.cycles-pp.__alloc_skb
      0.00            +1.7        1.73 ±  8%  perf-profile.self.cycles-pp.memset_erms
      1.26            +1.8        3.08        perf-profile.self.cycles-pp.kmem_cache_alloc_node_trace





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Oliver Sang


View attachment "config-5.2.0-05754-g6471384af2a65" of type "text/plain" (200966 bytes)

View attachment "job-script" of type "text/plain" (7232 bytes)

View attachment "job.yaml" of type "text/plain" (4851 bytes)

View attachment "reproduce" of type "text/plain" (3026 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ