lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20200618005117.GB5279@shao2-debian>
Date:   Thu, 18 Jun 2020 08:51:17 +0800
From:   kernel test robot <rong.a.chen@...el.com>
To:     Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
Cc:     Linus Torvalds <torvalds@...ux-foundation.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org
Subject: [pipe] 566d136289: stress-ng.tee.ops_per_sec -84.7% regression

Greeting,

FYI, we noticed a -84.7% regression of stress-ng.tee.ops_per_sec due to commit:


commit: 566d136289dc57816ac290de87a9a0f7d9bd3cbb ("pipe: Fix pipe_full() test in opipe_prep().")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

in testcase: stress-ng
on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
with following parameters:

	nr_threads: 100%
	disk: 1HDD
	sc_pid_max: 4194304
	testtime: 1s
	class: scheduler
	cpufreq_governor: performance
	ucode: 0xb000038


In addition to that, the commit also has significant impact on the following tests:

If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install job.yaml  # job file is attached in this email
        bin/lkp run     job.yaml

=========================================================================================
class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/sc_pid_max/tbox_group/testcase/testtime/ucode:
  scheduler/gcc-7/performance/1HDD/x86_64-rhel-7.6/100%/debian-x86_64-20191114.cgz/4194304/lkp-bdw-ep6/stress-ng/1s/0xb000038

commit: 
  115a54162a ("Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs")
  566d136289 ("pipe: Fix pipe_full() test in opipe_prep().")

115a54162a6c0d0e 566d136289dc57816ac290de87a 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    549404 ±  9%     +18.1%     648858 ± 13%  stress-ng.affinity.ops
    543194 ±  9%     +18.6%     644077 ± 13%  stress-ng.affinity.ops_per_sec
   2376013 ± 23%     -84.7%     363639 ± 19%  stress-ng.tee.ops_per_sec
    668966 ± 29%     -32.7%     450302 ± 17%  cpuidle.POLL.usage
   7750890 ± 15%     +32.9%   10298845 ±  8%  meminfo.DirectMap2M
     18658 ±  2%      +8.8%      20308 ±  2%  proc-vmstat.nr_page_table_pages
  31083952 ±  6%      -9.0%   28278712 ±  6%  proc-vmstat.pgfault
      1519 ± 19%     -27.3%       1104 ±  8%  sched_debug.cfs_rq:/.util_avg.max
     95.69 ± 12%     +20.9%     115.73 ±  8%  sched_debug.cfs_rq:/.util_est_enqueued.stddev
      2305 ±  2%     +15.1%       2654 ±  2%  slabinfo.task_group.active_objs
      2305 ±  2%     +15.1%       2654 ±  2%  slabinfo.task_group.num_objs
   1501755            -9.1%    1365035 ± 10%  perf-stat.i.context-switches
  11721777 ±  4%      -7.8%   10804056        perf-stat.i.iTLB-loads
      0.32 ±  2%      -3.6%       0.31 ±  2%  perf-stat.i.ipc
     11.41 ±  3%      +8.7%      12.39 ±  3%  perf-stat.overall.MPKI
     64.50            +1.9       66.38        perf-stat.overall.iTLB-load-miss-rate%
 6.252e+08            +5.1%  6.574e+08        perf-stat.ps.cache-references
   1386747 ±  2%      -8.8%    1264746 ±  8%  perf-stat.ps.context-switches
      9976 ± 16%     +32.1%      13175 ± 21%  softirqs.CPU14.RCU
     29368           +35.9%      39921 ± 25%  softirqs.CPU14.TIMER
     12696 ±171%     -99.7%      42.75 ± 16%  softirqs.CPU15.NET_RX
      8568 ±  2%     +16.8%      10007 ± 15%  softirqs.CPU18.SCHED
     12197 ± 10%     -16.1%      10229 ±  5%  softirqs.CPU24.RCU
     11990 ± 12%     +23.1%      14759 ±  2%  softirqs.CPU25.RCU
     11449            -9.8%      10328 ±  3%  softirqs.CPU42.RCU
     11154 ±  5%      -9.9%      10051 ±  4%  softirqs.CPU43.RCU
      8637 ±  3%     +10.0%       9502 ±  5%  softirqs.CPU65.SCHED
     11075 ±  7%      -9.6%      10015 ±  7%  softirqs.CPU71.RCU
     11622 ±  3%     -12.2%      10204 ±  3%  softirqs.CPU77.RCU
     47890 ±  3%      +8.3%      51887 ±  7%  softirqs.NET_RX
     16041 ±172%     -99.8%      39.50 ± 40%  interrupts.35:IR-PCI-MSI.1572866-edge.eth0-TxRx-1
     18539 ± 29%    +244.4%      63844 ± 36%  interrupts.CPU13.RES:Rescheduling_interrupts
     43700 ± 43%     -62.7%      16294 ± 15%  interrupts.CPU14.RES:Rescheduling_interrupts
     16041 ±172%     -99.8%      39.50 ± 40%  interrupts.CPU15.35:IR-PCI-MSI.1572866-edge.eth0-TxRx-1
     26345 ± 40%    +167.3%      70408 ± 32%  interrupts.CPU18.RES:Rescheduling_interrupts
     20745 ± 15%    +229.0%      68259 ± 14%  interrupts.CPU2.RES:Rescheduling_interrupts
      4907 ± 12%     +14.6%       5621 ±  5%  interrupts.CPU20.CAL:Function_call_interrupts
     19478 ± 27%    +190.3%      56540 ± 34%  interrupts.CPU20.RES:Rescheduling_interrupts
     49136 ± 40%     -61.2%      19058 ± 14%  interrupts.CPU42.RES:Rescheduling_interrupts
     22949 ± 28%    +167.7%      61439 ± 23%  interrupts.CPU45.RES:Rescheduling_interrupts
      4897 ± 15%     +16.4%       5700 ± 11%  interrupts.CPU46.CAL:Function_call_interrupts
     28.00 ±122%    +914.3%     284.00 ±136%  interrupts.CPU46.IWI:IRQ_work_interrupts
     24257 ± 45%    +142.0%      58699 ± 38%  interrupts.CPU59.RES:Rescheduling_interrupts
      5.00 ± 92%   +3055.0%     157.75 ±148%  interrupts.CPU6.IWI:IRQ_work_interrupts
     22519 ± 21%    +190.0%      65311 ± 17%  interrupts.CPU60.RES:Rescheduling_interrupts
      2039 ± 10%     +45.4%       2964 ± 13%  interrupts.CPU64.TLB:TLB_shootdowns
     17409 ± 10%    +228.5%      57183 ± 28%  interrupts.CPU65.RES:Rescheduling_interrupts
     37695 ± 46%     -62.2%      14231 ± 14%  interrupts.CPU75.RES:Rescheduling_interrupts
      5079 ± 11%     -15.3%       4303 ±  9%  interrupts.CPU77.CAL:Function_call_interrupts
      2.13 ±  4%      -0.3        1.83 ± 12%  perf-profile.calltrace.cycles-pp.inode_sb_list_add.new_inode.ramfs_get_inode.ramfs_mknod.path_openat
      2.10 ±  4%      -0.3        1.80 ± 12%  perf-profile.calltrace.cycles-pp._raw_spin_lock.inode_sb_list_add.new_inode.ramfs_get_inode.ramfs_mknod
      2.29 ±  4%      -0.3        2.00 ± 11%  perf-profile.calltrace.cycles-pp.ramfs_mknod.path_openat.do_filp_open.do_sys_openat2.do_sys_open
      2.24 ±  4%      -0.3        1.94 ± 11%  perf-profile.calltrace.cycles-pp.new_inode.ramfs_get_inode.ramfs_mknod.path_openat.do_filp_open
      2.07 ±  4%      -0.3        1.77 ± 12%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.inode_sb_list_add.new_inode.ramfs_get_inode
      2.27 ±  4%      -0.3        1.98 ± 11%  perf-profile.calltrace.cycles-pp.ramfs_get_inode.ramfs_mknod.path_openat.do_filp_open.do_sys_openat2
      3.54 ±  3%      -0.3        3.25 ±  8%  perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
      3.55 ±  3%      -0.3        3.26 ±  8%  perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
      1.02 ±  4%      -0.1        0.88 ± 13%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.evict.do_unlinkat.do_syscall_64
      1.03 ±  4%      -0.1        0.90 ± 13%  perf-profile.calltrace.cycles-pp._raw_spin_lock.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.68 ±  5%      -0.1        0.63 ±  8%  perf-profile.calltrace.cycles-pp.fanotify_merge.fsnotify_add_event.fanotify_handle_event.fsnotify.do_sys_openat2
      3.14 ±  2%      +0.1        3.25 ±  3%  perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.eventfd_write.vfs_write
      3.22            +0.1        3.36 ±  3%  perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
      3.42 ±  2%      +0.2        3.64 ±  2%  perf-profile.calltrace.cycles-pp.mmap64
      1.59 ±  5%      +0.2        1.82 ±  8%  perf-profile.calltrace.cycles-pp.__shmem_file_setup.shmem_zero_setup.mmap_region.do_mmap.vm_mmap_pgoff
      3.34 ±  2%      +0.2        3.57 ±  3%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64
      1.59 ±  5%      +0.2        1.82 ±  8%  perf-profile.calltrace.cycles-pp.shmem_zero_setup.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff
      3.34 ±  2%      +0.2        3.57 ±  3%  perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      1.45 ±  5%      +0.2        1.68 ±  9%  perf-profile.calltrace.cycles-pp.shmem_get_inode.__shmem_file_setup.shmem_zero_setup.mmap_region.do_mmap
      3.24 ±  2%      +0.2        3.48 ±  2%  perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      3.23 ±  2%      +0.2        3.46 ±  3%  perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
      2.31 ±  4%      -0.3        2.01 ± 11%  perf-profile.children.cycles-pp.ramfs_mknod
      2.29 ±  4%      -0.3        1.99 ± 11%  perf-profile.children.cycles-pp.ramfs_get_inode
      0.07 ±  7%      -0.0        0.04 ± 58%  perf-profile.children.cycles-pp.percpu_counter_add_batch
      0.15 ±  3%      -0.0        0.14 ±  6%  perf-profile.children.cycles-pp.native_flush_tlb_one_user
      0.10 ± 12%      +0.0        0.12 ±  8%  perf-profile.children.cycles-pp.vma_interval_tree_remove
      0.09 ± 27%      +0.0        0.12 ± 16%  perf-profile.children.cycles-pp.move_queued_task
      0.13 ± 21%      +0.0        0.17 ± 11%  perf-profile.children.cycles-pp.__set_cpus_allowed_ptr
      0.06 ± 65%      +0.1        0.11 ± 18%  perf-profile.children.cycles-pp.rwsem_wake
      3.59 ±  2%      +0.2        3.80 ±  2%  perf-profile.children.cycles-pp.vm_mmap_pgoff
      3.51 ±  2%      +0.2        3.72 ±  2%  perf-profile.children.cycles-pp.ksys_mmap_pgoff
      3.50 ±  2%      +0.2        3.73 ±  2%  perf-profile.children.cycles-pp.mmap64
      1.59 ±  5%      +0.2        1.82 ±  8%  perf-profile.children.cycles-pp.__shmem_file_setup
      1.59 ±  5%      +0.2        1.82 ±  8%  perf-profile.children.cycles-pp.shmem_zero_setup
      7.90            +0.2        8.13 ±  2%  perf-profile.children.cycles-pp.task_work_run
      0.15 ±  3%      -0.0        0.14 ±  6%  perf-profile.self.cycles-pp.native_flush_tlb_one_user


                                                                                
                               stress-ng.tee.ops_per_sec                        
                                                                                
    4e+06 +-----------------------------------------------------------------+   
          |      .+...                                                      |   
  3.5e+06 |... ..     +....+..    .+...+..                                  |   
    3e+06 |-+ +               . ..        .                     +           |   
          |                    +           +           +.      : +          |   
  2.5e+06 |-+                               :        ..  ..   :   +         |   
          |                                  :      .      .  :    +        |   
    2e+06 |-+                                :    .+         :      +       |   
          |                                   : ..          +               |   
  1.5e+06 |-+                                  +                            |   
    1e+06 |-+                                                               |   
          |                                                                 |   
   500000 |-+                                                       O       |   
          |   O   O   O    O   O   O   O   O   O   O   O    O   O       O   |   
        0 +-----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
=========================================================================================
class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/sc_pid_max/tbox_group/testcase/testtime/ucode:
  scheduler/gcc-9/performance/1HDD/x86_64-rhel-7.6/100%/debian-x86_64-20191114.cgz/4194304/lkp-bdw-ep6/stress-ng/10s/0xb000038

commit: 
  115a54162a ("Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs")
  566d136289 ("pipe: Fix pipe_full() test in opipe_prep().")

115a54162a6c0d0e 566d136289dc57816ac290de87a 
---------------- --------------------------- 
       fail:runs  %reproduction    fail:runs
           |             |             |    
          0:1           -1%           0:1     perf-profile.children.cycles-pp.error_entry
          0:1           -1%           0:1     perf-profile.self.cycles-pp.error_entry



***************************************************************************************************
lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory




Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


Thanks,
Rong Chen


View attachment "config-5.7.0-rc6-00038-g566d136289dc5" of type "text/plain" (202622 bytes)

View attachment "job-script" of type "text/plain" (7824 bytes)

View attachment "job.yaml" of type "text/plain" (5371 bytes)

View attachment "reproduce" of type "text/plain" (438 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ