lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20191006113847.GF17687@shao2-debian>
Date:   Sun, 6 Oct 2019 19:38:47 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Eric Dumazet <edumazet@...gle.com>
Cc:     "David S. Miller" <davem@...emloft.net>,
        Soheil Hassas Yeganeh <soheil@...gle.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>, lkp@...org
Subject: [tcp] 1a9914884d: netpipe.bigger_5M_Mbps.avg -8.0% regression

Greeting,

FYI, we noticed a -8.0% regression of netpipe.bigger_5M_Mbps.avg due to commit:


commit: 1a9914884db5138682032cf69f2d55739f236c80 ("tcp: batch calls to sk_flush_backlog()")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: netpipe
on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory
with following parameters:

	test: tcp
	cluster: cs-localhost
	cpufreq_governor: performance
	ucode: 0x21

test-description: NetPIPE is a protocol independent performance tool that encapsulates the best of ttcp and netperf and visually represents the network performance under a variety of conditions.
test-url: http://bitspjoule.org/netpipe/



If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
cluster/compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/ucode:
  cs-localhost/gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-2019-05-14.cgz/lkp-ivb-d04/tcp/netpipe/0x21

commit: 
  fcc32a2165 ("liquidio: Use pcie_flr() instead of reimplementing it")
  1a9914884d ("tcp: batch calls to sk_flush_backlog()")

fcc32a21655e26d3 1a9914884db5138682032cf69f2 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          1:4          -25%            :4     dmesg.WARNING:at#for_ip_interrupt_entry/0x
         %stddev     %change         %stddev
             \          |                \  
     26920            -8.0%      24780        netpipe.bigger_5M_Mbps.avg
    150.28            +9.4%     164.43        netpipe.time.elapsed_time
    150.28            +9.4%     164.43        netpipe.time.elapsed_time.max
     86.00            +2.3%      88.00        netpipe.time.percent_of_cpu_this_job_got
    127.29           +12.6%     143.34        netpipe.time.system_time
   3699450            +1.8%    3764243        netpipe.time.voluntary_context_switches
      0.00 ± 33%      +0.0        0.01 ±115%  mpstat.cpu.all.iowait%
      5539            +1.7%       5633        proc-vmstat.nr_shmem
    225822            +7.2%     242028        proc-vmstat.pgfault
     55.25            -2.7%      53.75        vmstat.cpu.id
    107545            -6.2%     100878        vmstat.system.cs
     13936            -5.1%      13223        vmstat.system.in
    981178 ±  9%     -11.6%     867496 ±  2%  cpuidle.C1E.usage
    219332 ±  6%     +69.2%     371070 ± 33%  cpuidle.C6.usage
    159555           +18.7%     189397 ±  5%  cpuidle.POLL.time
     26366 ±  4%     +65.5%      43647 ±  8%  cpuidle.POLL.usage
     40000 ±  5%     +11.1%      44434 ±  4%  softirqs.CPU0.SCHED
     69896 ±  4%     +11.0%      77578 ±  3%  softirqs.CPU1.TIMER
     35079 ±  2%     +16.4%      40815        softirqs.CPU3.SCHED
    151984           +10.7%     168187        softirqs.SCHED
      1584            +3.3%       1636        turbostat.Avg_MHz
      6.39 ±  5%      -1.2        5.20 ±  3%  turbostat.C1%
    981176 ±  9%     -11.6%     867494 ±  2%  turbostat.C1E
      9.48 ± 16%      -2.1        7.39 ±  2%  turbostat.C1E%
    219331 ±  6%     +69.2%     371070 ± 33%  turbostat.C6
      0.92 ± 59%     -69.1%       0.28 ± 11%  turbostat.CPU%c6
      4025 ±  2%      +8.4%       4361        interrupts.CAL:Function_call_interrupts
     22995 ± 11%     -27.9%      16570 ± 13%  interrupts.CPU1.RES:Rescheduling_interrupts
    926.25 ±  3%     +21.0%       1121        interrupts.CPU2.CAL:Function_call_interrupts
     20381 ± 12%     -48.3%      10532 ± 24%  interrupts.CPU2.RES:Rescheduling_interrupts
    477890 ±  2%     +13.4%     541884 ±  3%  interrupts.CPU3.LOC:Local_timer_interrupts
    960.50 ±166%    +565.2%       6389 ± 39%  interrupts.CPU3.NMI:Non-maskable_interrupts
    960.50 ±166%    +565.2%       6389 ± 39%  interrupts.CPU3.PMI:Performance_monitoring_interrupts
     84789 ± 15%     -34.0%      55991 ±  4%  interrupts.RES:Rescheduling_interrupts
    108.00 ± 11%     -19.0%      87.50 ± 16%  interrupts.TLB:TLB_shootdowns
    554079 ± 21%     -27.0%     404433 ±  6%  sched_debug.cfs_rq:/.load.max
    225.69 ± 17%     -21.7%     176.67 ± 13%  sched_debug.cfs_rq:/.runnable_load_avg.avg
    470.75 ± 20%     -41.0%     277.75 ±  8%  sched_debug.cfs_rq:/.runnable_load_avg.max
    170.37 ± 18%     -47.8%      88.97 ± 20%  sched_debug.cfs_rq:/.runnable_load_avg.stddev
    458733 ± 30%     -30.4%     319089 ±  3%  sched_debug.cfs_rq:/.runnable_weight.max
   1490943 ±  4%     -10.4%    1335260 ±  5%  sched_debug.cpu.nr_switches.min
     -7.83           +38.3%     -10.83        sched_debug.cpu.nr_uninterruptible.min
      6.19 ±  8%     +30.3%       8.06 ± 13%  sched_debug.cpu.nr_uninterruptible.stddev
   1482291 ±  4%     -10.5%    1326215 ±  5%  sched_debug.cpu.sched_count.min
    726955 ±  4%      -9.0%     661412 ±  5%  sched_debug.cpu.sched_goidle.min
    743551 ±  4%     -10.3%     666695 ±  5%  sched_debug.cpu.ttwu_count.min
    260937 ±  8%     -19.0%     211342 ± 18%  sched_debug.cpu.ttwu_local.max
 4.265e+08 ±  8%     -20.0%  3.411e+08 ± 14%  perf-stat.i.branch-instructions
 2.499e+08 ±  4%     -21.6%  1.959e+08 ± 17%  perf-stat.i.cache-references
    109510            -6.6%     102314        perf-stat.i.context-switches
   3211371 ±  4%     -15.4%    2716219 ±  9%  perf-stat.i.dTLB-load-misses
 9.824e+08 ±  5%     -20.9%  7.772e+08 ± 16%  perf-stat.i.dTLB-loads
    853729 ±  4%     -16.1%     715879 ± 10%  perf-stat.i.dTLB-store-misses
 7.672e+08 ±  4%     -21.1%  6.056e+08 ± 17%  perf-stat.i.dTLB-stores
    712689 ±  3%     -17.8%     586172 ± 14%  perf-stat.i.iTLB-load-misses
    324859 ±  4%     -16.0%     272850 ± 15%  perf-stat.i.iTLB-loads
 2.062e+09 ±  8%     -20.0%  1.649e+09 ± 14%  perf-stat.i.instructions
      0.30 ±  3%     -18.0%       0.25 ± 12%  perf-stat.i.ipc
     27.16            +3.6       30.73 ±  3%  perf-stat.overall.cache-miss-rate%
  4.24e+08 ±  8%     -20.0%  3.393e+08 ± 14%  perf-stat.ps.branch-instructions
 2.484e+08 ±  3%     -21.5%  1.949e+08 ± 17%  perf-stat.ps.cache-references
    108872            -6.5%     101753        perf-stat.ps.context-switches
   3192441 ±  4%     -15.4%    2701404 ±  9%  perf-stat.ps.dTLB-load-misses
 9.766e+08 ±  5%     -20.8%   7.73e+08 ± 16%  perf-stat.ps.dTLB-loads
    848715 ±  4%     -16.1%     711942 ± 10%  perf-stat.ps.dTLB-store-misses
 7.627e+08 ±  4%     -21.0%  6.023e+08 ± 17%  perf-stat.ps.dTLB-stores
    708519 ±  3%     -17.7%     583007 ± 14%  perf-stat.ps.iTLB-load-misses
    322966 ±  4%     -16.0%     271378 ± 15%  perf-stat.ps.iTLB-loads
  2.05e+09 ±  8%     -20.0%   1.64e+09 ± 14%  perf-stat.ps.instructions
     40.55 ±  6%      -9.9       30.64 ± 22%  perf-profile.calltrace.cycles-pp.write
     40.38 ±  6%      -9.9       30.52 ± 22%  perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
     40.37 ±  6%      -9.9       30.52 ± 22%  perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
     40.35 ±  6%      -9.8       30.51 ± 22%  perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
     40.31 ±  6%      -9.8       30.47 ± 22%  perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
     40.30 ±  6%      -9.8       30.46 ± 22%  perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
     40.31 ±  6%      -9.8       30.47 ± 22%  perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
     40.38 ±  6%      -9.8       30.54 ± 22%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
     40.38 ±  6%      -9.8       30.54 ± 22%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
     40.23 ±  6%      -9.8       30.43 ± 22%  perf-profile.calltrace.cycles-pp.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
     26.63 ±  6%      -6.2       20.48 ± 23%  perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.sock_write_iter
     26.27 ±  6%      -6.2       20.12 ± 23%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg
     26.44 ±  6%      -6.1       20.32 ± 23%  perf-profile.calltrace.cycles-pp.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
     26.41 ± 10%      -5.6       20.84 ± 21%  perf-profile.calltrace.cycles-pp.read
     25.61 ± 10%      -5.3       20.27 ± 21%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
     25.56 ± 10%      -5.3       20.23 ± 21%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
     24.89 ± 10%      -5.1       19.84 ± 21%  perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
     24.75 ± 10%      -5.0       19.74 ± 21%  perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
     24.15 ± 10%      -4.8       19.37 ± 21%  perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
     24.00 ± 10%      -4.7       19.30 ± 21%  perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
     23.77 ± 10%      -4.6       19.17 ± 21%  perf-profile.calltrace.cycles-pp.inet_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
     23.71 ± 10%      -4.6       19.14 ± 21%  perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.sock_read_iter.new_sync_read.vfs_read
      9.44 ±  7%      -2.6        6.86 ± 23%  perf-profile.calltrace.cycles-pp.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.sock_write_iter
      8.53 ±  8%      -2.4        6.17 ± 23%  perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      8.02 ±  8%      -2.3        5.72 ± 23%  perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked.tcp_sendmsg
      7.66 ±  7%      -2.2        5.46 ± 23%  perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.tcp_sendmsg_locked
      7.28 ±  8%      -2.1        5.22 ± 23%  perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
      7.19 ±  9%      -2.0        5.23 ± 27%  perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit
      7.22 ±  9%      -1.9        5.28 ± 27%  perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
      7.09 ± 10%      -1.9        5.16 ± 27%  perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output
      7.05 ± 10%      -1.9        5.14 ± 27%  perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2
      6.89 ± 10%      -1.8        5.04 ± 28%  perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq.__local_bh_enable_ip
      6.75 ± 10%      -1.8        4.92 ± 28%  perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack.do_softirq
      6.56 ± 10%      -1.8        4.80 ± 27%  perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.do_softirq_own_stack
      5.54 ± 12%      -1.6        3.95 ± 23%  perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog
      5.49 ± 12%      -1.6        3.92 ± 23%  perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core
      5.38 ± 12%      -1.5        3.86 ± 23%  perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv
      5.62 ± 13%      -1.5        4.12 ± 27%  perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action
      1.93 ± 34%      -1.2        0.78 ± 63%  perf-profile.calltrace.cycles-pp.printk.irq_work_run_list.irq_work_run.smp_irq_work_interrupt.irq_work_interrupt
      1.93 ± 34%      -1.2        0.78 ± 63%  perf-profile.calltrace.cycles-pp.vprintk_emit.printk.irq_work_run_list.irq_work_run.smp_irq_work_interrupt
      1.93 ± 34%      -1.1        0.78 ± 63%  perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.printk.irq_work_run_list.irq_work_run
      1.83 ± 35%      -1.1        0.75 ± 64%  perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.printk.irq_work_run_list
      1.80 ± 35%      -1.1        0.75 ± 65%  perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.printk
      1.75 ± 35%      -1.0        0.73 ± 65%  perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
      1.75 ± 35%      -1.0        0.73 ± 65%  perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
      4.32 ± 10%      -1.0        3.31 ± 25%  perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver
      3.44 ± 10%      -0.9        2.51 ± 22%  perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv
      3.40 ± 10%      -0.9        2.48 ± 23%  perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established
      2.73 ±  9%      -0.8        1.92 ± 25%  perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
      2.72 ±  9%      -0.8        1.92 ± 25%  perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock
      2.69 ±  9%      -0.8        1.91 ± 25%  perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.__wake_up_common
      2.32 ± 17%      -0.8        1.55 ± 20%  perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_read_iter.new_sync_read
      2.55 ± 10%      -0.8        1.79 ± 25%  perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
      1.97 ± 18%      -0.7        1.24 ± 19%  perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg.sock_read_iter
      2.17 ± 16%      -0.7        1.45 ± 21%  perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg.sock_read_iter.new_sync_read
      1.59 ± 16%      -0.7        0.87 ± 22%  perf-profile.calltrace.cycles-pp.__sk_flush_backlog.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.sock_write_iter
      1.54 ± 17%      -0.7        0.86 ± 21%  perf-profile.calltrace.cycles-pp.__release_sock.__sk_flush_backlog.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
      1.93 ± 16%      -0.6        1.29 ± 20%  perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg.sock_read_iter
      1.46 ± 16%      -0.6        0.84 ± 20%  perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.__sk_flush_backlog.tcp_sendmsg_locked
      1.47 ± 16%      -0.6        0.85 ± 21%  perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.__sk_flush_backlog.tcp_sendmsg_locked.tcp_sendmsg
      1.75 ± 18%      -0.6        1.16 ± 19%  perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg
      1.98 ±  9%      -0.6        1.42 ± 27%  perf-profile.calltrace.cycles-pp.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
      0.86 ± 13%      -0.5        0.32 ±100%  perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.schedule_timeout.wait_woken
      1.83 ±  8%      -0.5        1.31 ± 28%  perf-profile.calltrace.cycles-pp.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task
      1.50 ± 22%      -0.5        1.00 ± 16%  perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg
      1.28 ± 19%      -0.5        0.80 ± 21%  perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.__sk_flush_backlog
      0.76 ± 14%      -0.5        0.28 ±100%  perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule.schedule_timeout
      1.70 ±  7%      -0.5        1.22 ± 27%  perf-profile.calltrace.cycles-pp.arch_stack_walk.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
      1.02 ± 18%      -0.3        0.76 ± 22%  perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
     66.30 ±  7%     -15.0       51.28 ± 21%  perf-profile.children.cycles-pp.do_syscall_64
     66.34 ±  7%     -15.0       51.33 ± 21%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
     40.55 ±  6%      -9.9       30.65 ± 22%  perf-profile.children.cycles-pp.write
     40.38 ±  6%      -9.8       30.54 ± 22%  perf-profile.children.cycles-pp.ksys_write
     40.31 ±  6%      -9.8       30.47 ± 22%  perf-profile.children.cycles-pp.sock_write_iter
     40.37 ±  6%      -9.8       30.52 ± 22%  perf-profile.children.cycles-pp.vfs_write
     40.30 ±  6%      -9.8       30.46 ± 22%  perf-profile.children.cycles-pp.tcp_sendmsg
     40.35 ±  6%      -9.8       30.52 ± 22%  perf-profile.children.cycles-pp.new_sync_write
     40.31 ±  6%      -9.8       30.48 ± 22%  perf-profile.children.cycles-pp.sock_sendmsg
     40.26 ±  6%      -9.8       30.44 ± 22%  perf-profile.children.cycles-pp.tcp_sendmsg_locked
     26.63 ±  6%      -6.2       20.48 ± 23%  perf-profile.children.cycles-pp._copy_from_iter_full
     26.45 ±  6%      -6.1       20.32 ± 23%  perf-profile.children.cycles-pp.copyin
     26.46 ± 10%      -5.6       20.88 ± 21%  perf-profile.children.cycles-pp.read
     24.89 ± 10%      -5.1       19.84 ± 21%  perf-profile.children.cycles-pp.ksys_read
     24.75 ± 10%      -5.0       19.75 ± 21%  perf-profile.children.cycles-pp.vfs_read
     24.15 ± 10%      -4.8       19.38 ± 21%  perf-profile.children.cycles-pp.new_sync_read
     24.00 ± 10%      -4.7       19.31 ± 21%  perf-profile.children.cycles-pp.sock_read_iter
     23.77 ± 10%      -4.6       19.17 ± 21%  perf-profile.children.cycles-pp.inet_recvmsg
     23.72 ± 10%      -4.6       19.14 ± 21%  perf-profile.children.cycles-pp.tcp_recvmsg
     11.05 ±  8%      -3.1        7.98 ± 22%  perf-profile.children.cycles-pp.__tcp_transmit_skb
     10.25 ±  7%      -2.9        7.32 ± 22%  perf-profile.children.cycles-pp.__ip_queue_xmit
      9.68 ±  7%      -2.8        6.91 ± 22%  perf-profile.children.cycles-pp.ip_output
      9.76 ±  7%      -2.6        7.20 ± 22%  perf-profile.children.cycles-pp.tcp_write_xmit
      9.03 ±  6%      -2.5        6.49 ± 22%  perf-profile.children.cycles-pp.ip_finish_output2
      7.64 ± 11%      -1.9        5.73 ± 22%  perf-profile.children.cycles-pp.__softirqentry_text_start
      7.49 ±  9%      -1.8        5.67 ± 23%  perf-profile.children.cycles-pp.__local_bh_enable_ip
      7.42 ±  9%      -1.8        5.60 ± 23%  perf-profile.children.cycles-pp.do_softirq
      7.30 ± 10%      -1.8        5.53 ± 23%  perf-profile.children.cycles-pp.do_softirq_own_stack
      7.12 ± 10%      -1.7        5.41 ± 23%  perf-profile.children.cycles-pp.net_rx_action
      6.95 ± 10%      -1.7        5.26 ± 24%  perf-profile.children.cycles-pp.process_backlog
      6.77 ± 10%      -1.6        5.12 ± 24%  perf-profile.children.cycles-pp.__netif_receive_skb_one_core
      5.91 ± 11%      -1.5        4.37 ± 23%  perf-profile.children.cycles-pp.tcp_rcv_established
      5.95 ± 11%      -1.5        4.40 ± 23%  perf-profile.children.cycles-pp.tcp_v4_do_rcv
      6.02 ±  9%      -1.4        4.63 ± 24%  perf-profile.children.cycles-pp.ip_local_deliver
      5.92 ±  9%      -1.4        4.57 ± 23%  perf-profile.children.cycles-pp.ip_local_deliver_finish
      5.86 ±  9%      -1.3        4.53 ± 23%  perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
      5.76 ±  9%      -1.3        4.44 ± 23%  perf-profile.children.cycles-pp.tcp_v4_rcv
      3.64 ± 10%      -1.0        2.66 ± 22%  perf-profile.children.cycles-pp.__wake_up_common
      3.61 ± 10%      -1.0        2.64 ± 23%  perf-profile.children.cycles-pp.try_to_wake_up
      2.90 ±  9%      -0.9        2.03 ± 25%  perf-profile.children.cycles-pp.ttwu_do_activate
      2.89 ±  9%      -0.9        2.03 ± 25%  perf-profile.children.cycles-pp.activate_task
      2.85 ±  8%      -0.8        2.02 ± 25%  perf-profile.children.cycles-pp.enqueue_task_fair
      2.71 ±  9%      -0.8        1.90 ± 25%  perf-profile.children.cycles-pp.enqueue_entity
      2.54 ± 10%      -0.8        1.75 ± 24%  perf-profile.children.cycles-pp.__sched_text_start
      2.32 ± 17%      -0.8        1.55 ± 20%  perf-profile.children.cycles-pp.sk_wait_data
      1.97 ± 18%      -0.7        1.24 ± 19%  perf-profile.children.cycles-pp.wait_woken
      1.59 ± 16%      -0.7        0.87 ± 22%  perf-profile.children.cycles-pp.__sk_flush_backlog
      1.58 ± 18%      -0.7        0.92 ± 22%  perf-profile.children.cycles-pp.__release_sock
      1.34 ± 14%      -0.6        0.71 ± 15%  perf-profile.children.cycles-pp.__dev_queue_xmit
      2.10 ±  9%      -0.6        1.49 ± 27%  perf-profile.children.cycles-pp.__account_scheduler_latency
      1.94 ±  8%      -0.6        1.37 ± 28%  perf-profile.children.cycles-pp.stack_trace_save_tsk
      1.81 ±  6%      -0.5        1.28 ± 28%  perf-profile.children.cycles-pp.arch_stack_walk
      1.33 ± 18%      -0.5        0.83 ± 20%  perf-profile.children.cycles-pp.tcp_ack
      0.82 ± 21%      -0.4        0.43 ± 21%  perf-profile.children.cycles-pp.dev_hard_start_xmit
      1.18 ±  9%      -0.4        0.82 ± 26%  perf-profile.children.cycles-pp.__kfree_skb
      0.75 ± 28%      -0.4        0.39 ± 25%  perf-profile.children.cycles-pp.loopback_xmit
      0.87 ± 12%      -0.3        0.57 ± 11%  perf-profile.children.cycles-pp.dequeue_task_fair
      1.06 ± 18%      -0.3        0.78 ± 20%  perf-profile.children.cycles-pp.tcp_clean_rtx_queue
      0.79 ± 14%      -0.3        0.52 ± 13%  perf-profile.children.cycles-pp.dequeue_entity
      1.09 ± 10%      -0.3        0.82 ± 14%  perf-profile.children.cycles-pp.menu_select
      0.31 ± 19%      -0.2        0.11 ± 25%  perf-profile.children.cycles-pp.tcp_schedule_loss_probe
      0.33 ± 19%      -0.2        0.15 ± 28%  perf-profile.children.cycles-pp.__netif_receive_skb_core
      0.36 ± 13%      -0.2        0.18 ± 26%  perf-profile.children.cycles-pp.security_file_permission
      0.27 ± 15%      -0.2        0.12 ± 58%  perf-profile.children.cycles-pp.unwind_get_return_address
      0.48 ± 15%      -0.1        0.34 ±  9%  perf-profile.children.cycles-pp.update_load_avg
      0.44 ± 16%      -0.1        0.30 ± 20%  perf-profile.children.cycles-pp.security_sock_rcv_skb
      0.46 ± 19%      -0.1        0.33 ± 20%  perf-profile.children.cycles-pp.sk_filter_trim_cap
      0.31 ± 31%      -0.1        0.18 ± 30%  perf-profile.children.cycles-pp.update_process_times
      0.25 ± 22%      -0.1        0.13 ± 40%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.20 ± 18%      -0.1        0.08 ± 65%  perf-profile.children.cycles-pp.avc_has_perm
      0.17 ± 17%      -0.1        0.04 ± 59%  perf-profile.children.cycles-pp.file_has_perm
      0.40 ± 16%      -0.1        0.28 ± 24%  perf-profile.children.cycles-pp.selinux_socket_sock_rcv_skb
      0.23 ± 13%      -0.1        0.12 ± 23%  perf-profile.children.cycles-pp.selinux_parse_skb
      0.22 ± 15%      -0.1        0.11 ± 57%  perf-profile.children.cycles-pp.__kernel_text_address
      0.34 ± 18%      -0.1        0.23 ± 26%  perf-profile.children.cycles-pp._raw_spin_lock
      0.18 ± 10%      -0.1        0.07 ± 58%  perf-profile.children.cycles-pp.kernel_text_address
      0.26 ± 17%      -0.1        0.16 ± 33%  perf-profile.children.cycles-pp.__unwind_start
      0.22 ± 10%      -0.1        0.12 ± 25%  perf-profile.children.cycles-pp.update_cfs_group
      0.57 ±  6%      -0.1        0.47 ± 13%  perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
      0.20 ± 10%      -0.1        0.10 ± 10%  perf-profile.children.cycles-pp.validate_xmit_skb
      0.26 ± 23%      -0.1        0.17 ± 12%  perf-profile.children.cycles-pp.entry_SYSCALL_64
      0.26 ± 19%      -0.1        0.17 ± 25%  perf-profile.children.cycles-pp.skb_release_all
      0.24 ± 17%      -0.1        0.15 ± 20%  perf-profile.children.cycles-pp.skb_release_head_state
      0.35 ± 12%      -0.1        0.27 ± 17%  perf-profile.children.cycles-pp.ktime_get
      0.33 ± 11%      -0.1        0.26 ± 16%  perf-profile.children.cycles-pp.get_next_timer_interrupt
      0.14 ±  9%      -0.1        0.07 ± 71%  perf-profile.children.cycles-pp.tick_nohz_idle_exit
      0.10 ± 18%      -0.1        0.03 ±102%  perf-profile.children.cycles-pp.tcp_queue_rcv
      0.15 ± 24%      -0.1        0.08 ± 29%  perf-profile.children.cycles-pp.pick_next_task_idle
      0.20 ±  7%      -0.1        0.14 ± 26%  perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
      0.18 ±  8%      -0.1        0.12 ± 36%  perf-profile.children.cycles-pp.tcp_mstamp_refresh
      0.12 ± 22%      -0.1        0.06 ± 60%  perf-profile.children.cycles-pp.cfb_imageblit
      0.10 ± 27%      -0.1        0.04 ± 63%  perf-profile.children.cycles-pp.cpuacct_charge
      0.17 ± 26%      -0.1        0.12 ± 12%  perf-profile.children.cycles-pp.update_rq_clock
      0.12 ± 22%      -0.1        0.07 ± 59%  perf-profile.children.cycles-pp.drm_fb_helper_cfb_imageblit
      0.09 ± 35%      -0.1        0.04 ±113%  perf-profile.children.cycles-pp.in_sched_functions
      0.12 ± 18%      -0.0        0.07 ± 20%  perf-profile.children.cycles-pp.scheduler_tick
      0.13 ± 33%      -0.0        0.08 ± 19%  perf-profile.children.cycles-pp.ns_to_timespec64
      0.08 ± 17%      -0.0        0.04 ± 60%  perf-profile.children.cycles-pp.netif_skb_features
      0.11 ± 17%      -0.0        0.08 ±  8%  perf-profile.children.cycles-pp.__update_load_avg_se
      0.03 ±100%      +0.0        0.07 ±  7%  perf-profile.children.cycles-pp.tcp_rbtree_insert
      0.07 ± 50%      +0.1        0.15 ± 32%  perf-profile.children.cycles-pp.__do_execve_file
      0.07 ± 50%      +0.1        0.15 ± 34%  perf-profile.children.cycles-pp.__x64_sys_execve
      0.07 ± 50%      +0.1        0.15 ± 31%  perf-profile.children.cycles-pp.execve
      0.05 ± 63%      +0.1        0.18 ± 33%  perf-profile.children.cycles-pp.tcp_add_backlog
      0.69 ±  6%      -0.2        0.46 ± 25%  perf-profile.self.cycles-pp.unwind_next_frame
      0.32 ± 18%      -0.2        0.15 ± 28%  perf-profile.self.cycles-pp.__netif_receive_skb_core
      0.44 ± 15%      -0.1        0.30 ± 24%  perf-profile.self.cycles-pp.__check_object_size
      0.48 ±  3%      -0.1        0.35 ± 21%  perf-profile.self.cycles-pp.do_syscall_64
      0.16 ± 32%      -0.1        0.04 ±108%  perf-profile.self.cycles-pp.dequeue_entity
      0.20 ± 21%      -0.1        0.08 ± 65%  perf-profile.self.cycles-pp.avc_has_perm
      0.25 ± 21%      -0.1        0.14 ± 29%  perf-profile.self.cycles-pp.ip_finish_output2
      0.23 ± 10%      -0.1        0.12 ± 23%  perf-profile.self.cycles-pp.selinux_parse_skb
      0.32 ± 14%      -0.1        0.21 ± 30%  perf-profile.self.cycles-pp.menu_select
      0.29 ± 14%      -0.1        0.18 ± 20%  perf-profile.self.cycles-pp.skb_release_data
      0.22 ± 10%      -0.1        0.11 ± 28%  perf-profile.self.cycles-pp.update_cfs_group
      0.22 ± 23%      -0.1        0.12 ± 49%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.32 ± 15%      -0.1        0.22 ± 23%  perf-profile.self.cycles-pp._raw_spin_lock
      0.14 ± 13%      -0.1        0.04 ± 58%  perf-profile.self.cycles-pp.tcp_clean_rtx_queue
      0.24 ± 24%      -0.1        0.15 ± 13%  perf-profile.self.cycles-pp.entry_SYSCALL_64
      0.22 ± 11%      -0.1        0.14 ± 21%  perf-profile.self.cycles-pp.tcp_v4_rcv
      0.17 ±  8%      -0.1        0.09 ± 17%  perf-profile.self.cycles-pp.wait_woken
      0.10 ± 18%      -0.1        0.03 ±102%  perf-profile.self.cycles-pp.tcp_queue_rcv
      0.11 ± 33%      -0.1        0.04 ±103%  perf-profile.self.cycles-pp.new_sync_read
      0.10 ± 15%      -0.1        0.03 ±100%  perf-profile.self.cycles-pp.validate_xmit_skb
      0.15 ± 19%      -0.1        0.08 ± 31%  perf-profile.self.cycles-pp.process_backlog
      0.19 ± 18%      -0.1        0.13 ± 16%  perf-profile.self.cycles-pp.__skb_datagram_iter
      0.12 ± 22%      -0.1        0.06 ± 60%  perf-profile.self.cycles-pp.cfb_imageblit
      0.15 ± 19%      -0.1        0.09 ± 34%  perf-profile.self.cycles-pp.__softirqentry_text_start
      0.10 ± 25%      -0.1        0.04 ± 63%  perf-profile.self.cycles-pp.cpuacct_charge
      0.13 ± 33%      -0.0        0.08 ± 19%  perf-profile.self.cycles-pp.ns_to_timespec64
      0.10 ± 30%      -0.0        0.06 ± 11%  perf-profile.self.cycles-pp.ip_rcv_finish_core
      0.03 ±100%      +0.0        0.07 ±  7%  perf-profile.self.cycles-pp.tcp_rbtree_insert


                                                                                
                             netpipe.time.system_time                           
                                                                                
  155 +-+-----O-------------------------------------------------------------+   
      |                           O O                                       |   
  150 +-+          O                                                        |   
      |O        O   O                O                                      |   
  145 +-O    O O      O  O OOOO OO O  O O      O                            |   
      O  O  O    OO     O O            O  OOOOO                             |   
  140 +-+ O            O                                                    |   
      |                                                                     |   
  135 +-+                                                                   |   
      |                                                                     |   
  130 +-+     +                     +          +      +    +                |   
      |+++ .++ ++   +.  ++  +++.+ ++ ++++.++ ++ +++.++ ++++ +.+++++++++.++++|   
  125 +-+ +      +++  ++  ++     +          +                               |   
      |                                                                     |   
  120 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                     netpipe.time.percent_of_cpu_this_job_got                   
                                                                                
    89 +-+--------OO--------------------------------------------------------+   
       |                                                                    |   
  88.5 +-+                                                                  |   
    88 OOOOOO O OO  OOOO OOOOOOOOOOO OOOOOOOOOO O                           |   
       |                                                                    |   
  87.5 +-+                                                                  |   
       |                                                                    |   
    87 +-+     O                                                            |   
       |                                                                    |   
  86.5 +-+                                                                  |   
    86 +-+ +  ++ +++ +++.++++ +  + +.+ + +  + +.+ + +++++++    +++++++.+++++|   
       |:  :  :: : : :      : :  : : : : :  : : : : :      :   :            |   
  85.5 +-+ :::  ::  ::       ::: :::  ::::: :::  ::::      :   :            |   
       | :: ::  :   :        : :: :   : : :: :   : :        : :             |   
    85 +-+------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             netpipe.time.elapsed_time                          
                                                                                
  180 +-+-------------------------------------------------------------------+   
      |       O                                                             |   
  175 +-+                         O O                                       |   
      |            O                                                        |   
  170 +O+       O                                                           |   
      |             O       O O  O O OO        O                            |   
  165 O-OO  OO O      O OOOO O  O      OO O O O                             |   
      |   O      OO                        O O                              |   
  160 +-+              O                                                    |   
      |                                                                     |   
  155 +-+                                                  +                |   
      |       +++                   +++        +++   ++    :+.+     +     + |   
  150 +-++ .++   : ++.  ++  +++.+ ++   ++.++ ++   +.+  ++++    +++++ ++.++ +|   
      |   +      ++   ++  ++     +          +                               |   
  145 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                           netpipe.time.elapsed_time.max                        
                                                                                
  180 +-+-------------------------------------------------------------------+   
      |       O                                                             |   
  175 +-+                         O O                                       |   
      |            O                                                        |   
  170 +O+       O                                                           |   
      |             O       O O  O O OO        O                            |   
  165 O-OO  OO O      O OOOO O  O      OO O O O                             |   
      |   O      OO                        O O                              |   
  160 +-+              O                                                    |   
      |                                                                     |   
  155 +-+                                                  +                |   
      |       +++                   +++        +++   ++    :+.+     +     + |   
  150 +-++ .++   : ++.  ++  +++.+ ++   ++.++ ++   +.+  ++++    +++++ ++.++ +|   
      |   +      ++   ++  ++     +          +                               |   
  145 +-+-------------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             netpipe.bigger_5M_Mbps.avg                         
                                                                                
  27500 +-+-----------------------------------------------------------------+   
  27000 +-+        ++  +++++.+                                              |   
        |++ +++.+ +  ++       +++ ++++ +++.+ ++++ +++  +.+++   ++++++.+ ++++|   
  26500 +-++     +               +    +     +    +   ++     +++        +    |   
  26000 +-+                                                                 |   
        |                                                                   |   
  25500 +-+             O                                                   |   
  25000 +-+OO      OO    OOO O    O O   O   O OO                            |   
  24500 +-O  O        OO      OO      OO   O O  O                           |   
        OO    O  OO              O       O                                  |   
  24000 +-+          O          O                                           |   
  23500 +-+                                                                 |   
        |                          O O                                      |   
  23000 +-+                                                                 |   
  22500 +-+-----O-----------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample



Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.3.0-rc3-00727-g1a9914884db51" of type "text/plain" (199638 bytes)

View attachment "job-script" of type "text/plain" (7447 bytes)

View attachment "job.yaml" of type "text/plain" (4921 bytes)

View attachment "reproduce" of type "text/plain" (292 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ