lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20210907150757.GE17617@xsang-OptiPlex-9020>
Date:   Tue, 7 Sep 2021 23:07:57 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Vasily Averin <vvs@...tuozzo.com>
Cc:     Linus Torvalds <torvalds@...ux-foundation.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Alexander Viro <viro@...iv.linux.org.uk>,
        Alexey Dobriyan <adobriyan@...il.com>,
        Andrei Vagin <avagin@...il.com>,
        Borislav Petkov <bp@...en8.de>, Borislav Petkov <bp@...e.de>,
        Christian Brauner <christian.brauner@...ntu.com>,
        Dmitry Safonov <0x7f454c46@...il.com>,
        "Eric W. Biederman" <ebiederm@...ssion.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        "H. Peter Anvin" <hpa@...or.com>, Ingo Molnar <mingo@...hat.com>,
        "J. Bruce Fields" <bfields@...ldses.org>,
        Jeff Layton <jlayton@...nel.org>, Jens Axboe <axboe@...nel.dk>,
        Jiri Slaby <jirislaby@...nel.org>,
        Johannes Weiner <hannes@...xchg.org>,
        Kirill Tkhai <ktkhai@...tuozzo.com>,
        Michal Hocko <mhocko@...nel.org>,
        Oleg Nesterov <oleg@...hat.com>, Roman Gushchin <guro@...com>,
        Serge Hallyn <serge@...lyn.com>, Tejun Heo <tj@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Vladimir Davydov <vdavydov.dev@...il.com>,
        Yutian Yang <nglaive@...il.com>,
        Zefan Li <lizefan.x@...edance.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        lkp@...el.com, ying.huang@...el.com, feng.tang@...el.com,
        zhengjun.xing@...ux.intel.com
Subject: [memcg]  0f12156dff:  will-it-scale.per_process_ops -33.6% regression



Greeting,

FYI, we noticed a -33.6% regression of will-it-scale.per_process_ops due to commit:


commit: 0f12156dff2862ac54235fc72703f18770769042 ("memcg: enable accounting for file lock caches")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: will-it-scale
on test machine: 104 threads 2 sockets Skylake with 192G memory
with following parameters:

	nr_task: 50%
	mode: process
	test: lock1
	cpufreq_governor: performance
	ucode: 0x2006a0a

test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale

In addition to that, the commit also has significant impact on the following tests:

+------------------+---------------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops -40.9% regression                  |
| test machine     | 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory |
| test parameters  | cpufreq_governor=performance                                                    |
|                  | mode=process                                                                    |
|                  | nr_task=50%                                                                     |
|                  | test=lock1                                                                      |
|                  | ucode=0xd000280                                                                 |
+------------------+---------------------------------------------------------------------------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp install                job.yaml  # job file is attached in this email
        bin/lkp split-job --compatible job.yaml  # generate the yaml file for lkp run
        bin/lkp run                    generated-yaml-file

=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/lock1/will-it-scale/0x2006a0a

commit: 
  b655843444 ("memcg: enable accounting for pollfd and select bits arrays")
  0f12156dff ("memcg: enable accounting for file lock caches")

b655843444152c0a 0f12156dff2862ac54235fc7270 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
  65855410           -33.6%   43722413 ±  5%  will-it-scale.52.processes
   1266449           -33.6%     840815 ±  5%  will-it-scale.per_process_ops
  65855410           -33.6%   43722413 ±  5%  will-it-scale.workload
    141099 ±  3%      +8.2%     152680 ±  2%  meminfo.Active
    140875 ±  3%      +8.2%     152456 ±  2%  meminfo.Active(anon)
     28.79            +6.7       35.47        mpstat.cpu.all.sys%
     20.49            -6.7       13.76 ±  5%  mpstat.cpu.all.usr%
    138801 ±  3%      +8.4%     150456 ±  2%  numa-meminfo.node1.Active
    138689 ±  3%      +8.4%     150381 ±  2%  numa-meminfo.node1.Active(anon)
     34681 ±  3%      +8.3%      37570 ±  2%  numa-vmstat.node1.nr_active_anon
     34681 ±  3%      +8.3%      37570 ±  2%  numa-vmstat.node1.nr_zone_active_anon
      4480 ±  3%     -85.9%     632.67 ± 12%  slabinfo.Acpi-Parse.active_objs
      4480 ±  3%     -85.9%     632.67 ± 12%  slabinfo.Acpi-Parse.num_objs
   2957456 ±214%     -96.5%     104939 ±  5%  turbostat.C1
    103.99            +5.8%     110.04        turbostat.RAMWatt
     19.83           -34.5%      13.00 ±  4%  vmstat.cpu.us
      2487 ±  2%      +4.4%       2596        vmstat.system.cs
     35219 ±  3%      +8.2%      38114 ±  2%  proc-vmstat.nr_active_anon
     10977            +1.8%      11171        proc-vmstat.nr_mapped
     40640 ±  3%      +7.6%      43745 ±  2%  proc-vmstat.nr_shmem
     35219 ±  3%      +8.2%      38114 ±  2%  proc-vmstat.nr_zone_active_anon
     53061 ±  3%      +6.5%      56488 ±  3%  proc-vmstat.pgactivate
      0.02 ± 19%     -24.0%       0.01 ± 18%  perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      9.26 ± 21%     -39.8%       5.58 ± 14%  perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
    172.83 ± 11%     -29.4%     122.00 ±  5%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk
    127.50 ±  4%     +47.8%     188.50 ±  7%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode
      1080 ± 19%     +66.6%       1799 ± 13%  perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      7204 ±  3%     -12.0%       6337        perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      7204 ±  3%     -12.0%       6337        perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      7209 ±  3%     -12.0%       6342        perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      6825 ±  4%     -11.5%       6036        perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
      9.26 ± 21%     -39.8%       5.57 ± 14%  perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork
      7204 ±  3%     -12.0%       6337        perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      7204 ±  3%     -12.0%       6337        perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      0.05 ±  7%    +751.3%       0.40 ±180%  perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown]
      7209 ±  3%     -12.0%       6342        perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      6825 ±  4%     -11.5%       6036        perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
    827.67 ±  9%     -16.3%     692.50 ±  4%  interrupts.CPU13.CAL:Function_call_interrupts
      3623 ± 54%     +66.8%       6045 ± 33%  interrupts.CPU26.NMI:Non-maskable_interrupts
      3623 ± 54%     +66.8%       6045 ± 33%  interrupts.CPU26.PMI:Performance_monitoring_interrupts
    239.50 ± 18%     -33.1%     160.33 ± 38%  interrupts.CPU41.RES:Rescheduling_interrupts
    261.33 ± 18%     -54.4%     119.17 ± 49%  interrupts.CPU42.RES:Rescheduling_interrupts
      7894           -30.2%       5511 ± 37%  interrupts.CPU45.NMI:Non-maskable_interrupts
      7894           -30.2%       5511 ± 37%  interrupts.CPU45.PMI:Performance_monitoring_interrupts
      3374 ± 40%     +90.6%       6432 ± 23%  interrupts.CPU58.NMI:Non-maskable_interrupts
      3374 ± 40%     +90.6%       6432 ± 23%  interrupts.CPU58.PMI:Performance_monitoring_interrupts
    272.83 ±  8%     -36.4%     173.50 ± 40%  interrupts.CPU6.RES:Rescheduling_interrupts
      2890 ± 37%     +61.0%       4652 ± 37%  interrupts.CPU76.NMI:Non-maskable_interrupts
      2890 ± 37%     +61.0%       4652 ± 37%  interrupts.CPU76.PMI:Performance_monitoring_interrupts
    272.67 ±  9%     -29.0%     193.67 ± 27%  interrupts.CPU81.RES:Rescheduling_interrupts
      3475 ± 47%    +109.3%       7273 ± 12%  interrupts.CPU93.NMI:Non-maskable_interrupts
      3475 ± 47%    +109.3%       7273 ± 12%  interrupts.CPU93.PMI:Performance_monitoring_interrupts
      3054 ± 37%     +91.7%       5857 ± 38%  interrupts.CPU96.NMI:Non-maskable_interrupts
      3054 ± 37%     +91.7%       5857 ± 38%  interrupts.CPU96.PMI:Performance_monitoring_interrupts
      0.05 ±  5%   +1601.9%       0.79 ± 20%  perf-stat.i.MPKI
     12.52 ±  2%     +20.9       33.42 ±  4%  perf-stat.i.cache-miss-rate%
    647128 ±  2%   +4855.0%   32065121 ±  8%  perf-stat.i.cache-misses
   5130146 ±  2%   +1777.3%   96305803 ±  9%  perf-stat.i.cache-references
      2302            +6.1%       2443        perf-stat.i.context-switches
    259518 ±  2%     -98.1%       4827 ±  6%  perf-stat.i.cycles-between-cache-misses
      0.17            -0.1        0.11        perf-stat.i.dTLB-load-miss-rate%
  65644478           -33.6%   43563819 ±  5%  perf-stat.i.dTLB-load-misses
     51892           -27.4%      37675 ±  3%  perf-stat.i.dTLB-store-misses
  68181011           -33.4%   45405593 ±  5%  perf-stat.i.iTLB-load-misses
      1933           +58.2%       3058 ±  3%  perf-stat.i.instructions-per-iTLB-miss
     81.87 ± 12%    +782.2%     722.30 ± 16%  perf-stat.i.metric.K/sec
    126326 ±  3%   +6119.4%    7856755 ±  2%  perf-stat.i.node-load-misses
     26893 ±  6%   +3382.5%     936567 ±  3%  perf-stat.i.node-loads
     87.04           +12.8       99.79        perf-stat.i.node-store-miss-rate%
     33838 ±  3%  +28351.8%    9627739        perf-stat.i.node-store-misses
      0.04 ±  2%   +1709.7%       0.71 ± 14%  perf-stat.overall.MPKI
      0.52            -0.0        0.49 ±  3%  perf-stat.overall.branch-miss-rate%
     12.65 ±  2%     +20.7       33.35 ±  4%  perf-stat.overall.cache-miss-rate%
    222708 ±  2%     -98.0%       4560 ±  8%  perf-stat.overall.cycles-between-cache-misses
      0.17            -0.1        0.10        perf-stat.overall.dTLB-load-miss-rate%
      0.00            -0.0        0.00 ±  2%  perf-stat.overall.dTLB-store-miss-rate%
      1929           +56.0%       3010        perf-stat.overall.instructions-per-iTLB-miss
     82.22            +7.1       89.34        perf-stat.overall.node-load-miss-rate%
     78.75           +21.2       99.91        perf-stat.overall.node-store-miss-rate%
    603681           +56.8%     946452        perf-stat.overall.path-length
    649626 ±  2%   +4816.5%   31938893 ±  8%  perf-stat.ps.cache-misses
   5137402 ±  2%   +1767.5%   95941923 ±  9%  perf-stat.ps.cache-references
      2313            +5.8%       2448        perf-stat.ps.context-switches
  65424262           -33.6%   43419292 ±  5%  perf-stat.ps.dTLB-load-misses
     51749           -27.4%      37573 ±  3%  perf-stat.ps.dTLB-store-misses
  67951468           -33.4%   45252180 ±  5%  perf-stat.ps.iTLB-load-misses
    126213 ±  3%   +6102.7%    7828694 ±  2%  perf-stat.ps.node-load-misses
     27281 ±  6%   +3321.9%     933553 ±  3%  perf-stat.ps.node-loads
     33783 ±  3%  +28302.8%    9595407        perf-stat.ps.node-store-misses
     15207 ± 10%     -34.0%      10039 ± 12%  softirqs.CPU0.RCU
     16968 ± 17%     -36.3%      10810 ± 19%  softirqs.CPU1.RCU
     17526 ± 19%     -36.4%      11142 ± 14%  softirqs.CPU10.RCU
     17232 ± 16%     -48.1%       8946 ± 13%  softirqs.CPU100.RCU
     15770 ± 16%     -43.7%       8874 ± 28%  softirqs.CPU101.RCU
     14396 ± 15%     -44.6%       7969 ± 13%  softirqs.CPU102.RCU
     18909 ±  5%     -46.4%      10130 ± 17%  softirqs.CPU103.RCU
     17822 ± 23%     -45.3%       9744 ± 22%  softirqs.CPU11.RCU
     18056 ± 22%     -52.5%       8582 ± 22%  softirqs.CPU12.RCU
     14576 ± 17%     -33.3%       9723 ± 15%  softirqs.CPU13.RCU
     18836 ± 15%     -47.1%       9961 ±  8%  softirqs.CPU14.RCU
     20542 ± 17%     -54.3%       9394 ± 11%  softirqs.CPU15.RCU
     17485 ± 19%     -42.0%      10141 ± 19%  softirqs.CPU16.RCU
     17387 ± 17%     -37.8%      10807 ± 13%  softirqs.CPU17.RCU
     18473 ± 15%     -49.9%       9250 ± 18%  softirqs.CPU18.RCU
     19232 ± 19%     -44.1%      10751 ± 10%  softirqs.CPU19.RCU
     17052 ± 21%     -42.9%       9744 ± 22%  softirqs.CPU2.RCU
     17330 ± 17%     -43.5%       9797 ± 16%  softirqs.CPU20.RCU
     18624 ± 15%     -43.9%      10445 ± 24%  softirqs.CPU21.RCU
     18206 ± 18%     -44.4%      10123 ± 26%  softirqs.CPU22.RCU
     18047 ± 12%     -44.5%      10022 ± 23%  softirqs.CPU23.RCU
     19351 ± 18%     -47.7%      10115 ± 21%  softirqs.CPU24.RCU
     18629 ± 17%     -51.4%       9057 ± 18%  softirqs.CPU25.RCU
     15355 ± 14%     -44.8%       8480 ± 13%  softirqs.CPU26.RCU
     15129 ± 14%     -39.7%       9117 ± 16%  softirqs.CPU27.RCU
     14744 ± 10%     -39.0%       8996 ± 21%  softirqs.CPU28.RCU
     13973 ± 11%     -39.4%       8470 ± 18%  softirqs.CPU29.RCU
     35779 ± 10%     -27.5%      25930 ± 30%  softirqs.CPU29.SCHED
     18703 ± 16%     -43.4%      10577 ± 19%  softirqs.CPU3.RCU
     17000 ± 19%     -40.8%      10057 ± 21%  softirqs.CPU30.RCU
     18602 ± 16%     -40.6%      11040 ± 15%  softirqs.CPU31.RCU
     17242 ± 23%     -44.0%       9662 ± 19%  softirqs.CPU32.RCU
     17841 ± 15%     -43.1%      10144 ± 21%  softirqs.CPU33.RCU
     17867 ± 14%     -44.6%       9890 ± 21%  softirqs.CPU34.RCU
     19083 ± 16%     -46.7%      10177 ± 15%  softirqs.CPU35.RCU
     19161 ± 11%     -49.8%       9616 ± 24%  softirqs.CPU36.RCU
     19335 ± 15%     -47.7%      10106 ± 16%  softirqs.CPU37.RCU
     20460 ± 11%     -54.3%       9341 ± 17%  softirqs.CPU38.RCU
     10641 ± 58%    +117.0%      23087 ± 43%  softirqs.CPU38.SCHED
     20165 ±  8%     -53.5%       9381 ± 17%  softirqs.CPU39.RCU
     20158 ±  9%     -47.3%      10630 ± 20%  softirqs.CPU4.RCU
     18503 ± 20%     -40.7%      10980 ± 20%  softirqs.CPU40.RCU
     18917 ± 11%     -51.1%       9241 ± 17%  softirqs.CPU41.RCU
     19989 ±  9%     -57.3%       8544 ± 12%  softirqs.CPU42.RCU
     10393 ± 58%    +155.8%      26585 ± 30%  softirqs.CPU42.SCHED
     16499 ± 11%     -45.5%       8991 ± 19%  softirqs.CPU43.RCU
     18599 ± 16%     -49.3%       9433 ± 24%  softirqs.CPU44.RCU
     20721 ±  8%     -56.4%       9035 ± 14%  softirqs.CPU45.RCU
      6210 ± 26%    +213.4%      19464 ± 52%  softirqs.CPU45.SCHED
     16912 ± 21%     -43.6%       9547 ± 24%  softirqs.CPU46.RCU
     19085 ± 14%     -49.9%       9571 ± 15%  softirqs.CPU47.RCU
     16524 ± 21%     -46.7%       8815 ± 20%  softirqs.CPU48.RCU
     18590 ± 13%     -52.0%       8921 ± 11%  softirqs.CPU49.RCU
     19473 ± 15%     -47.6%      10198 ± 12%  softirqs.CPU5.RCU
     19807 ± 16%     -51.7%       9574 ± 20%  softirqs.CPU50.RCU
     14621 ±  7%     -44.6%       8100 ± 14%  softirqs.CPU51.RCU
     21579 ±  9%     -49.4%      10911 ± 16%  softirqs.CPU52.RCU
     17758 ± 25%     -50.6%       8778 ± 25%  softirqs.CPU53.RCU
     18495 ± 15%     -47.1%       9776 ± 18%  softirqs.CPU54.RCU
     16729 ± 20%     -46.4%       8963 ± 20%  softirqs.CPU55.RCU
     15031 ± 10%     -44.4%       8358 ± 18%  softirqs.CPU56.RCU
     15721 ± 17%     -42.2%       9080 ± 30%  softirqs.CPU57.RCU
     14757 ±  3%     -32.1%      10022 ± 23%  softirqs.CPU58.RCU
     37022 ±  9%     -41.4%      21679 ± 41%  softirqs.CPU58.SCHED
     16077 ±  6%     -41.1%       9466 ± 32%  softirqs.CPU59.RCU
     20064 ± 11%     -54.4%       9150 ± 18%  softirqs.CPU6.RCU
     14811 ± 18%     -46.8%       7880 ± 15%  softirqs.CPU60.RCU
     15191 ±  8%     -49.8%       7623 ± 14%  softirqs.CPU61.RCU
     16592 ± 13%     -49.0%       8455 ± 12%  softirqs.CPU62.RCU
     15392 ± 21%     -44.3%       8566 ± 17%  softirqs.CPU63.RCU
     18606 ± 16%     -52.5%       8837 ± 19%  softirqs.CPU65.RCU
     14770 ± 22%     -40.4%       8802 ± 29%  softirqs.CPU66.RCU
     14465 ± 11%     -32.3%       9797 ± 24%  softirqs.CPU67.RCU
     16993 ± 16%     -48.1%       8818 ± 12%  softirqs.CPU68.RCU
     18034 ± 14%     -52.9%       8497 ± 18%  softirqs.CPU69.RCU
     18628 ± 16%     -47.9%       9700 ± 18%  softirqs.CPU7.RCU
     16564 ± 17%     -42.4%       9545 ± 15%  softirqs.CPU70.RCU
     16333 ± 19%     -45.3%       8933 ± 25%  softirqs.CPU71.RCU
     17985 ± 19%     -52.0%       8624 ± 25%  softirqs.CPU72.RCU
     16575 ± 17%     -50.7%       8172 ± 15%  softirqs.CPU73.RCU
     17243 ± 18%     -48.5%       8875 ± 15%  softirqs.CPU74.RCU
     17293 ± 12%     -44.8%       9547 ± 15%  softirqs.CPU75.RCU
     16269 ± 20%     -46.7%       8673 ± 24%  softirqs.CPU76.RCU
     17326 ± 18%     -45.8%       9383 ± 22%  softirqs.CPU77.RCU
     18826 ± 18%     -50.6%       9292 ± 24%  softirqs.CPU78.RCU
     19893 ±  8%     -52.4%       9460 ± 18%  softirqs.CPU79.RCU
     18812 ± 19%     -43.6%      10616 ± 19%  softirqs.CPU8.RCU
     20001 ± 12%     -53.0%       9393 ± 14%  softirqs.CPU80.RCU
     20756 ±  5%     -53.9%       9562 ± 18%  softirqs.CPU81.RCU
     18840 ± 15%     -53.5%       8761 ± 17%  softirqs.CPU82.RCU
     17159 ± 21%     -52.3%       8189 ± 20%  softirqs.CPU83.RCU
     18671 ± 16%     -52.4%       8881 ± 19%  softirqs.CPU84.RCU
     18929 ± 11%     -55.5%       8421 ± 21%  softirqs.CPU85.RCU
     17559 ± 15%     -51.0%       8604 ± 13%  softirqs.CPU86.RCU
     17343 ± 14%     -53.0%       8154 ± 18%  softirqs.CPU87.RCU
     15481 ± 13%     -43.6%       8727 ± 14%  softirqs.CPU88.RCU
     16364 ± 14%     -43.9%       9187 ± 24%  softirqs.CPU89.RCU
     18492 ± 19%     -43.5%      10441 ± 14%  softirqs.CPU9.RCU
     14056 ±  7%     -38.3%       8666 ± 21%  softirqs.CPU90.RCU
     14189 ±  6%     -37.9%       8815 ± 24%  softirqs.CPU91.RCU
     15154 ± 14%     -48.1%       7867 ± 15%  softirqs.CPU92.RCU
     15589 ± 14%     -42.4%       8979 ± 25%  softirqs.CPU93.RCU
     14492 ± 10%     -34.4%       9504 ± 19%  softirqs.CPU94.RCU
     34198 ± 16%     -52.1%      16384 ± 44%  softirqs.CPU94.SCHED
     17203 ±  7%     -46.2%       9249 ± 16%  softirqs.CPU95.RCU
     15367 ± 11%     -44.0%       8601 ± 12%  softirqs.CPU96.RCU
     14328 ± 13%     -37.8%       8911 ± 27%  softirqs.CPU97.RCU
     16134 ± 16%     -46.3%       8672 ± 19%  softirqs.CPU98.RCU
     15282 ± 17%     -44.5%       8486 ± 18%  softirqs.CPU99.RCU
   1808149 ±  6%     -46.3%     971467 ± 15%  softirqs.RCU
     36250 ±  5%     +36.3%      49404 ±  2%  softirqs.TIMER
     17.47 ±  9%      -5.9       11.61 ±  9%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      9.44 ±  9%      -3.2        6.23 ±  9%  perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_fcntl64
      8.57 ±  9%      -2.9        5.64 ±  9%  perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_fcntl64
      2.27 ±  9%      -1.0        1.27 ± 29%  perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack.__libc_fcntl64
      2.68 ±  9%      -1.0        1.70 ±  9%  perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      2.65 ±  8%      -0.9        1.78 ±  9%  perf-profile.calltrace.cycles-pp.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      2.52 ±  9%      -0.9        1.65 ±  9%  perf-profile.calltrace.cycles-pp._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      0.80 ±  8%      -0.5        0.28 ±100%  perf-profile.calltrace.cycles-pp._raw_spin_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      1.34 ±  9%      -0.5        0.83 ±  9%  perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      1.62 ±  9%      -0.5        1.12 ± 10%  perf-profile.calltrace.cycles-pp.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl
      0.82 ± 10%      -0.4        0.38 ± 71%  perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      0.66 ± 11%      -0.4        0.28 ±100%  perf-profile.calltrace.cycles-pp.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      1.04 ±  9%      -0.3        0.69 ±  9%  perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64
      0.86 ±  9%      -0.3        0.59 ±  8%  perf-profile.calltrace.cycles-pp.__fget_light.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      0.00            +0.6        0.59 ± 11%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      1.06 ±  9%      +0.9        1.92 ± 12%  perf-profile.calltrace.cycles-pp.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      0.00            +0.9        0.92 ± 10%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      0.67 ±  9%      +1.0        1.68 ± 14%  perf-profile.calltrace.cycles-pp.kmem_cache_free.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      0.00            +1.4        1.38 ±  8%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      0.00            +1.8        1.79 ± 59%  perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk
      0.00            +2.2        2.18 ± 58%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl
      0.00            +2.2        2.21 ± 58%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      0.00            +2.4        2.35 ± 43%  perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode
      0.00            +2.8        2.80 ± 59%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk
      0.00            +2.8        2.84 ± 41%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait
      0.00            +2.9        2.86 ± 41%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      1.10 ±  9%      +2.9        3.98 ± 32%  perf-profile.calltrace.cycles-pp.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
      0.00            +3.0        2.99 ± 55%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      1.69 ±  9%      +3.8        5.49 ± 21%  perf-profile.calltrace.cycles-pp.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      3.39 ±  8%      +4.0        7.42 ± 22%  perf-profile.calltrace.cycles-pp.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
      3.00 ±  8%      +4.2        7.17 ± 23%  perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      0.00            +4.6        4.61 ± 29%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode
      0.00            +5.0        5.00 ± 27%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      6.48 ±  9%      +6.3       12.76 ± 12%  perf-profile.calltrace.cycles-pp.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      5.71 ±  9%      +6.5       12.21 ± 13%  perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      0.00            +7.3        7.35 ± 35%  perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock
     17.27 ±  9%      +8.5       25.78 ± 11%  perf-profile.calltrace.cycles-pp.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
     13.80 ±  9%      +9.6       23.35 ± 12%  perf-profile.calltrace.cycles-pp.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl
     28.29 ±  9%     +13.3       41.58 ± 12%  perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
     26.73 ±  9%     +13.8       40.53 ± 12%  perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
     23.74 ±  9%     +14.8       38.56 ± 13%  perf-profile.calltrace.cycles-pp.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
     17.55 ±  9%      -5.9       11.68 ±  9%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      9.74 ±  9%      -3.3        6.41 ±  9%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      9.02 ±  8%      -3.1        5.96 ±  9%  perf-profile.children.cycles-pp.__entry_text_start
      4.02 ±  9%      -1.5        2.53 ±  9%  perf-profile.children.cycles-pp.memset_erms
      2.58 ±  9%      -0.9        1.69 ±  9%  perf-profile.children.cycles-pp._copy_from_user
      2.68 ±  8%      -0.9        1.80 ±  9%  perf-profile.children.cycles-pp.security_file_lock
      2.02 ±  7%      -0.7        1.33 ±  9%  perf-profile.children.cycles-pp._raw_spin_lock
      1.40 ±  8%      -0.6        0.85 ± 10%  perf-profile.children.cycles-pp.___might_sleep
      1.67 ±  9%      -0.5        1.16 ± 10%  perf-profile.children.cycles-pp.common_file_perm
      1.48 ±  9%      -0.5        0.98 ±  9%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
      1.23 ±  9%      -0.4        0.81 ±  9%  perf-profile.children.cycles-pp.copy_user_generic_unrolled
      1.04 ±  9%      -0.4        0.67 ±  9%  perf-profile.children.cycles-pp.__might_sleep
      0.86 ± 10%      -0.3        0.56 ±  9%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      0.86 ±  9%      -0.3        0.59 ±  8%  perf-profile.children.cycles-pp.__fget_light
      0.76 ±  7%      -0.3        0.50 ±  8%  perf-profile.children.cycles-pp.apparmor_file_lock
      0.77 ±  8%      -0.3        0.51 ±  9%  perf-profile.children.cycles-pp.__cond_resched
      0.59 ±  8%      -0.2        0.38 ±  9%  perf-profile.children.cycles-pp.__might_fault
      0.66 ±  9%      -0.2        0.46 ±  9%  perf-profile.children.cycles-pp.locks_release_private
      0.56 ±  9%      -0.2        0.36 ±  9%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      0.58 ±  9%      -0.2        0.44 ± 10%  perf-profile.children.cycles-pp.locks_insert_lock_ctx
      0.69 ± 11%      -0.1        0.54 ±  9%  perf-profile.children.cycles-pp.locks_delete_lock_ctx
      0.56 ± 10%      -0.1        0.44 ±  8%  perf-profile.children.cycles-pp.locks_unlink_lock_ctx
      0.38 ±  8%      -0.1        0.26 ± 10%  perf-profile.children.cycles-pp.rcu_all_qs
      0.33 ± 11%      -0.1        0.21 ±  9%  perf-profile.children.cycles-pp.flock64_to_posix_lock
      0.35 ±  8%      -0.1        0.24 ± 10%  perf-profile.children.cycles-pp.aa_file_perm
      0.27 ± 21%      -0.1        0.16 ±  9%  perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
      0.23 ±  9%      -0.1        0.13 ±  8%  perf-profile.children.cycles-pp.__init_waitqueue_head
      0.31 ±  8%      -0.1        0.21 ±  9%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.24 ±  9%      -0.1        0.17 ±  6%  perf-profile.children.cycles-pp.testcase
      0.19 ± 11%      -0.1        0.12 ± 10%  perf-profile.children.cycles-pp.should_failslab
      0.15 ± 10%      -0.1        0.09 ± 10%  perf-profile.children.cycles-pp.fcntl@plt
      0.16 ±  7%      -0.1        0.10 ± 14%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
      0.13 ±  9%      -0.0        0.08 ±  5%  perf-profile.children.cycles-pp.security_file_fcntl
      0.12 ± 12%      -0.0        0.08 ±  8%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.14 ±  9%      -0.0        0.10 ± 10%  perf-profile.children.cycles-pp.locks_get_lock_context
      0.08 ± 10%      -0.0        0.03 ± 70%  perf-profile.children.cycles-pp.vfs_lock_file
      0.13 ±  8%      -0.0        0.10 ±  7%  perf-profile.children.cycles-pp.locks_delete_block
      0.10 ± 10%      -0.0        0.06 ± 14%  perf-profile.children.cycles-pp.__list_add_valid
      0.10 ± 10%      -0.0        0.06 ±  7%  perf-profile.children.cycles-pp.memset
      0.00            +0.1        0.08 ±  9%  perf-profile.children.cycles-pp.obj_cgroup_uncharge
      0.00            +0.2        0.21 ± 18%  perf-profile.children.cycles-pp.mem_cgroup_from_task
      0.00            +0.6        0.55 ±  9%  perf-profile.children.cycles-pp.rcu_read_unlock_strict
      0.00            +0.6        0.60 ±  9%  perf-profile.children.cycles-pp.refill_obj_stock
      1.10 ±  9%      +0.9        1.96 ± 12%  perf-profile.children.cycles-pp.locks_dispose_list
      0.00            +1.7        1.67 ± 10%  perf-profile.children.cycles-pp.mod_objcg_state
      0.00            +2.4        2.42 ± 35%  perf-profile.children.cycles-pp.propagate_protected_usage
      0.00            +2.5        2.47 ±  9%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
      0.00            +4.7        4.74 ± 36%  perf-profile.children.cycles-pp.page_counter_cancel
      0.00            +5.8        5.77 ± 35%  perf-profile.children.cycles-pp.page_counter_uncharge
      0.00            +5.8        5.82 ± 35%  perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
      0.00            +7.4        7.36 ± 35%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.00            +7.4        7.42 ± 35%  perf-profile.children.cycles-pp.obj_cgroup_charge_pages
      3.47 ±  9%      +7.8       11.27 ± 18%  perf-profile.children.cycles-pp.kmem_cache_free
      0.00            +8.1        8.05 ± 32%  perf-profile.children.cycles-pp.obj_cgroup_charge
     17.35 ±  9%      +8.5       25.81 ± 11%  perf-profile.children.cycles-pp.do_lock_file_wait
     14.01 ±  9%      +9.5       23.56 ± 12%  perf-profile.children.cycles-pp.posix_lock_inode
     10.00 ±  8%     +10.2       20.24 ± 14%  perf-profile.children.cycles-pp.locks_alloc_lock
      9.00 ±  8%     +10.7       19.67 ± 14%  perf-profile.children.cycles-pp.kmem_cache_alloc
     28.35 ±  9%     +13.3       41.61 ± 12%  perf-profile.children.cycles-pp.__x64_sys_fcntl
     26.89 ±  9%     +13.7       40.63 ± 12%  perf-profile.children.cycles-pp.do_fcntl
     23.92 ±  9%     +14.7       38.67 ± 13%  perf-profile.children.cycles-pp.fcntl_setlk
     17.12 ±  9%      -5.7       11.40 ±  9%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode
      9.72 ±  9%      -3.3        6.39 ±  9%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      7.81 ±  8%      -2.6        5.16 ±  9%  perf-profile.self.cycles-pp.__entry_text_start
      3.90 ±  9%      -1.4        2.45 ±  9%  perf-profile.self.cycles-pp.memset_erms
      1.97 ±  7%      -0.7        1.32 ± 10%  perf-profile.self.cycles-pp._raw_spin_lock
      1.78 ±  9%      -0.6        1.21 ±  9%  perf-profile.self.cycles-pp.__libc_fcntl64
      1.34 ±  8%      -0.5        0.82 ± 11%  perf-profile.self.cycles-pp.___might_sleep
      1.35 ±  9%      -0.4        0.94 ± 10%  perf-profile.self.cycles-pp.common_file_perm
      1.19 ±  9%      -0.4        0.79 ±  9%  perf-profile.self.cycles-pp.copy_user_generic_unrolled
      1.30 ±  8%      -0.3        0.96 ±  8%  perf-profile.self.cycles-pp.posix_lock_inode
      0.91 ±  9%      -0.3        0.58 ±  9%  perf-profile.self.cycles-pp.__might_sleep
      1.08 ±  9%      -0.3        0.76 ±  8%  perf-profile.self.cycles-pp.fcntl_setlk
      0.74 ±  8%      -0.3        0.43 ± 10%  perf-profile.self.cycles-pp.locks_alloc_lock
      0.79 ±  9%      -0.3        0.51 ± 10%  perf-profile.self.cycles-pp.syscall_enter_from_user_mode
      0.76 ±  9%      -0.3        0.49 ±  6%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.83 ±  9%      -0.3        0.56 ±  8%  perf-profile.self.cycles-pp.__fget_light
      0.70 ±  9%      -0.2        0.46 ±  8%  perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
      0.66 ±  7%      -0.2        0.43 ±  7%  perf-profile.self.cycles-pp.apparmor_file_lock
      0.64 ±  9%      -0.2        0.42 ±  9%  perf-profile.self.cycles-pp.locks_release_private
      0.52 ±  9%      -0.2        0.34 ±  9%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      0.47 ± 10%      -0.2        0.30 ±  9%  perf-profile.self.cycles-pp.do_lock_file_wait
      0.54 ±  9%      -0.2        0.38 ±  9%  perf-profile.self.cycles-pp.do_fcntl
      0.46 ±  9%      -0.2        0.30 ±  8%  perf-profile.self.cycles-pp.__x64_sys_fcntl
      0.40 ± 11%      -0.2        0.25 ± 11%  perf-profile.self.cycles-pp.do_syscall_64
      0.33 ± 11%      -0.1        0.19 ± 10%  perf-profile.self.cycles-pp.flock64_to_posix_lock
      0.38 ±  9%      -0.1        0.25 ±  9%  perf-profile.self.cycles-pp.__cond_resched
      0.25 ±  7%      -0.1        0.14 ± 10%  perf-profile.self.cycles-pp.locks_dispose_list
      0.24 ±  9%      -0.1        0.13 ±  8%  perf-profile.self.cycles-pp.testcase
      0.28 ±  7%      -0.1        0.19 ± 11%  perf-profile.self.cycles-pp.aa_file_perm
      0.22 ± 24%      -0.1        0.13 ±  8%  perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
      0.28 ±  8%      -0.1        0.18 ± 10%  perf-profile.self.cycles-pp.exit_to_user_mode_prepare
      0.25 ±  9%      -0.1        0.16 ± 10%  perf-profile.self.cycles-pp._copy_from_user
      0.26 ±  8%      -0.1        0.17 ± 10%  perf-profile.self.cycles-pp.rcu_all_qs
      0.18 ±  9%      -0.1        0.10 ± 10%  perf-profile.self.cycles-pp.__init_waitqueue_head
      0.15 ±  8%      -0.1        0.07 ± 14%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
      0.14 ±  9%      -0.1        0.08 ± 11%  perf-profile.self.cycles-pp.fcntl@plt
      0.12 ± 12%      -0.1        0.07 ±  7%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.18 ± 12%      -0.1        0.13 ±  8%  perf-profile.self.cycles-pp.security_file_lock
      0.08 ± 12%      -0.0        0.03 ±100%  perf-profile.self.cycles-pp.__list_add_valid
      0.12 ±  9%      -0.0        0.08 ± 11%  perf-profile.self.cycles-pp.locks_get_lock_context
      0.10 ± 10%      -0.0        0.06 ± 11%  perf-profile.self.cycles-pp.__might_fault
      0.10 ± 10%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.memset
      0.10 ±  8%      -0.0        0.06 ±  7%  perf-profile.self.cycles-pp.security_file_fcntl
      0.11 ±  8%      -0.0        0.08 ±  8%  perf-profile.self.cycles-pp.locks_delete_block
      0.00            +0.1        0.06 ±  7%  perf-profile.self.cycles-pp.obj_cgroup_uncharge
      0.00            +0.2        0.16 ± 24%  perf-profile.self.cycles-pp.mem_cgroup_from_task
      0.00            +0.3        0.34 ±  9%  perf-profile.self.cycles-pp.rcu_read_unlock_strict
      0.00            +0.6        0.56 ±  9%  perf-profile.self.cycles-pp.obj_cgroup_charge
      0.00            +0.6        0.58 ± 10%  perf-profile.self.cycles-pp.refill_obj_stock
      3.14 ±  8%      +1.3        4.46 ± 10%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.00            +1.7        1.65 ± 10%  perf-profile.self.cycles-pp.mod_objcg_state
      0.00            +2.2        2.17 ±  9%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current
      0.00            +2.4        2.40 ± 35%  perf-profile.self.cycles-pp.propagate_protected_usage
      0.00            +4.7        4.70 ± 36%  perf-profile.self.cycles-pp.page_counter_cancel
      0.00            +5.9        5.92 ± 35%  perf-profile.self.cycles-pp.page_counter_try_charge


                                                                                
                              will-it-scale.52.processes                        
                                                                                
    7e+07 +-----------------------------------------------------------------+   
          |   .+           +     +                  +     +                 |   
  6.5e+07 |+++  ++++++.++++ +++.+ +++++.+++++++.++++ ++.++ +++++.+++++++    |   
          |                                                                 |   
    6e+07 |-+                                                               |   
          |                                                                 |   
  5.5e+07 |-+                                                               |   
          |                                                                 |   
    5e+07 |-+            O                                                  |   
          |        OO   O   O      OOO  OO           O   O  OOO          O  |   
  4.5e+07 |O+O OOOO  O O   O O  OOO   O   OOOOO O O   O O OO   O  OOOO O    |   
          | O                 O                  O OO            O    O    O|   
    4e+07 |-+             O                                               O |   
          |                                                                 |   
  3.5e+07 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                             will-it-scale.per_process_ops                      
                                                                                
  1.3e+06 +-----------------------------------------------------------------+   
          |+++.+++++++.++++++++.+++++++.+++++++.+++++++.++++++++.+++++++    |   
  1.2e+06 |-+                                                               |   
          |                                                                 |   
          |                                                                 |   
  1.1e+06 |-+                                                               |   
          |                                                                 |   
    1e+06 |-+                                                               |   
          |              O                                                  |   
   900000 |-+       O       O      O O   O           O   O  O O          O  |   
          |O O OOO O O OO  O O  O O O   O  OOOO   O   O O OO O O  OOOO      |   
          |       O              O    O   O     OO OO            O    OO   O|   
   800000 |-O             O   O                                           O |   
          |                                                                 |   
   700000 +-----------------------------------------------------------------+   
                                                                                
                                                                                                                                                                
                                will-it-scale.workload                          
                                                                                
    7e+07 +-----------------------------------------------------------------+   
          |   .+           +     +                  +     +                 |   
  6.5e+07 |+++  ++++++.++++ +++.+ +++++.+++++++.++++ ++.++ +++++.+++++++    |   
          |                                                                 |   
    6e+07 |-+                                                               |   
          |                                                                 |   
  5.5e+07 |-+                                                               |   
          |                                                                 |   
    5e+07 |-+            O                                                  |   
          |        OO   O   O      OOO  OO           O   O  OOO          O  |   
  4.5e+07 |O+O OOOO  O O   O O  OOO   O   OOOOO O O   O O OO   O  OOOO O    |   
          | O                 O                  O OO            O    O    O|   
    4e+07 |-+             O                                               O |   
          |                                                                 |   
  3.5e+07 +-----------------------------------------------------------------+   
                                                                                
                                                                                
[*] bisect-good sample
[O] bisect-bad  sample

***************************************************************************************************
lkp-icl-2sp2: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
  gcc-9/performance/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp2/lock1/will-it-scale/0xd000280

commit: 
  b655843444 ("memcg: enable accounting for pollfd and select bits arrays")
  0f12156dff ("memcg: enable accounting for file lock caches")

b655843444152c0a 0f12156dff2862ac54235fc7270 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
 1.785e+08           -40.9%  1.055e+08 ±  2%  will-it-scale.64.processes
   2788911           -40.9%    1649196 ±  2%  will-it-scale.per_process_ops
 1.785e+08           -40.9%  1.055e+08 ±  2%  will-it-scale.workload
    366571 ±  7%     +15.2%     422206 ±  9%  numa-numastat.node0.local_node
      2807            +2.4%       2874        vmstat.system.cs
      0.02 ±  2%      -0.0        0.02 ±  2%  mpstat.cpu.all.soft%
      5.55            -2.4        3.15 ±  7%  mpstat.cpu.all.usr%
      5364 ±  2%     -85.5%     778.67 ± 13%  slabinfo.Acpi-Parse.active_objs
      5364 ±  2%     -85.5%     778.67 ± 13%  slabinfo.Acpi-Parse.num_objs
     49038            +5.1%      51541        proc-vmstat.nr_active_anon
     55209            +5.1%      58037        proc-vmstat.nr_shmem
     49038            +5.1%      51541        proc-vmstat.nr_zone_active_anon
    183.83 ± 28%    +320.7%     773.33 ±101%  interrupts.154:IR-PCI-MSI.25690117-edge.eth0-TxRx-5
    836.50 ± 10%     +46.3%       1223 ± 30%  interrupts.CPU103.CAL:Function_call_interrupts
      1348 ± 83%    +208.2%       4155 ± 50%  interrupts.CPU2.RES:Rescheduling_interrupts
      1089 ± 77%    +158.6%       2818 ± 53%  interrupts.CPU4.RES:Rescheduling_interrupts
    183.83 ± 28%    +320.7%     773.33 ±101%  interrupts.CPU5.154:IR-PCI-MSI.25690117-edge.eth0-TxRx-5
    806.17 ± 14%     +31.2%       1058 ± 30%  interrupts.CPU55.CAL:Function_call_interrupts
    723.17 ± 74%    +417.0%       3738 ± 82%  interrupts.CPU55.RES:Rescheduling_interrupts
      4847 ± 94%     -97.1%     138.17 ± 19%  interrupts.CPU86.NMI:Non-maskable_interrupts
      4847 ± 94%     -97.1%     138.17 ± 19%  interrupts.CPU86.PMI:Performance_monitoring_interrupts
      8069 ± 63%     -80.7%       1561 ±200%  interrupts.CPU99.NMI:Non-maskable_interrupts
      8069 ± 63%     -80.7%       1561 ±200%  interrupts.CPU99.PMI:Performance_monitoring_interrupts
      2345           -12.0%       2063        perf-sched.wait_and_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2345           -12.0%       2063        perf-sched.wait_and_delay.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
    262.68 ±  7%     -42.8%     150.13 ± 12%  perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
    561.17 ±  9%     -16.5%     468.33 ±  5%  perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
    183.33 ±  7%     +43.2%     262.50 ±  8%  perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode
    113.33 ± 11%     +78.7%     202.50 ±  4%  perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      7031           -12.0%       6185        perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      7031           -12.0%       6185        perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      7035           -12.0%       6189        perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      2345           -12.0%       2063        perf-sched.wait_time.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      2345           -12.0%       2063        perf-sched.wait_time.avg.ms.do_syslog.part.0.kmsg_read.vfs_read
      2.50 ±  7%     -47.6%       1.31 ± 18%  perf-sched.wait_time.avg.ms.rcu_gp_kthread.kthread.ret_from_fork
    262.68 ±  7%     -42.8%     150.13 ± 12%  perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll
      7031           -12.0%       6185        perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
      7031           -12.0%       6185        perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read
      7035           -12.0%       6189        perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read
      0.04 ± 22%   +1112.5%       0.50 ±128%  perf-sched.wait_time.max.ms.preempt_schedule_common.__cond_resched.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      0.02 ±  3%   +1574.6%       0.27 ±  6%  perf-stat.i.MPKI
 7.076e+10            -6.5%  6.615e+10 ±  2%  perf-stat.i.branch-instructions
      0.05            +0.0        0.08 ±  5%  perf-stat.i.branch-miss-rate%
  35955313 ±  2%     +45.6%   52354202 ±  5%  perf-stat.i.branch-misses
     10.31           +18.0       28.33 ±  8%  perf-stat.i.cache-miss-rate%
    492492         +4777.7%   24022051 ±  4%  perf-stat.i.cache-misses
   4620608         +1768.8%   86349315 ±  4%  perf-stat.i.cache-references
      2651            +2.5%       2716        perf-stat.i.context-switches
      0.47            +8.5%       0.51 ±  2%  perf-stat.i.cpi
    425867           -98.4%       6992 ±  5%  perf-stat.i.cycles-between-cache-misses
      0.00 ±  4%      +0.0        0.00 ± 11%  perf-stat.i.dTLB-load-miss-rate%
  1.03e+11            -3.8%  9.907e+10 ±  2%  perf-stat.i.dTLB-loads
      0.00            +0.0        0.00 ±  8%  perf-stat.i.dTLB-store-miss-rate%
 6.534e+10           -11.3%  5.796e+10 ±  2%  perf-stat.i.dTLB-stores
 3.516e+11            -7.2%  3.263e+11 ±  2%  perf-stat.i.instructions
      2.11            -7.1%       1.96 ±  2%  perf-stat.i.ipc
     37.50 ±  2%   +2090.1%     821.35 ±  3%  perf-stat.i.metric.K/sec
      1867            -6.7%       1743 ±  2%  perf-stat.i.metric.M/sec
    104632 ±  2%   +6707.0%    7122311 ±  3%  perf-stat.i.node-load-misses
     14377 ± 21%   +3209.4%     475820 ± 29%  perf-stat.i.node-loads
     45.30 ± 16%     +51.5       96.80        perf-stat.i.node-store-miss-rate%
     48602 ± 14%  +22917.3%   11186859 ±  6%  perf-stat.i.node-store-misses
     65851 ±  9%    +432.6%     350742 ±  9%  perf-stat.i.node-stores
      0.01         +1902.5%       0.27 ±  6%  perf-stat.overall.MPKI
      0.05            +0.0        0.08 ±  6%  perf-stat.overall.branch-miss-rate%
     10.65           +17.3       27.92 ±  8%  perf-stat.overall.cache-miss-rate%
      0.47            +7.7%       0.51 ±  2%  perf-stat.overall.cpi
    336119           -97.9%       6940 ±  4%  perf-stat.overall.cycles-between-cache-misses
      0.00 ±  4%      +0.0        0.00 ± 10%  perf-stat.overall.dTLB-load-miss-rate%
      0.00            +0.0        0.00 ±  4%  perf-stat.overall.dTLB-store-miss-rate%
      2.11            -7.1%       1.96 ±  2%  perf-stat.overall.ipc
     87.73 ±  2%      +6.0       93.76        perf-stat.overall.node-load-miss-rate%
     42.34 ± 13%     +54.6       96.95        perf-stat.overall.node-store-miss-rate%
    594415           +56.9%     932831        perf-stat.overall.path-length
  7.05e+10            -6.5%  6.592e+10 ±  2%  perf-stat.ps.branch-instructions
  35867377           +45.6%   52233576 ±  5%  perf-stat.ps.branch-misses
    493933         +4747.8%   23944727 ±  4%  perf-stat.ps.cache-misses
   4639810         +1756.0%   86115077 ±  4%  perf-stat.ps.cache-references
      2653            +2.6%       2721        perf-stat.ps.context-switches
 1.027e+11            -3.8%  9.872e+10 ±  2%  perf-stat.ps.dTLB-loads
  6.51e+10           -11.3%  5.775e+10 ±  2%  perf-stat.ps.dTLB-stores
 3.504e+11            -7.2%  3.252e+11 ±  2%  perf-stat.ps.instructions
    104869 ±  2%   +6668.1%    7097622 ±  3%  perf-stat.ps.node-load-misses
     14698 ± 21%   +3134.0%     475371 ± 29%  perf-stat.ps.node-loads
     48497 ± 14%  +22887.1%   11148161 ±  6%  perf-stat.ps.node-store-misses
     65971 ±  9%    +430.7%     350139 ±  8%  perf-stat.ps.node-stores
 1.061e+14            -7.2%  9.845e+13 ±  2%  perf-stat.total.instructions
     11738 ± 11%     -32.5%       7926 ± 19%  softirqs.CPU1.RCU
     12768 ± 12%     -39.4%       7741 ± 11%  softirqs.CPU10.RCU
      8853 ± 10%     -33.9%       5851 ±  8%  softirqs.CPU100.RCU
      9695 ±  6%     -40.1%       5811 ±  5%  softirqs.CPU101.RCU
      9277 ±  7%     -38.0%       5752 ±  7%  softirqs.CPU102.RCU
      9585 ±  8%     -33.5%       6376 ± 17%  softirqs.CPU103.RCU
      9838 ±  7%     -40.5%       5857 ±  9%  softirqs.CPU104.RCU
      9315 ±  8%     -40.6%       5529 ±  7%  softirqs.CPU105.RCU
      9661 ± 10%     -40.5%       5749 ±  5%  softirqs.CPU106.RCU
      9531 ±  6%     -35.6%       6138 ±  7%  softirqs.CPU108.RCU
      9472 ±  7%     -34.4%       6210 ± 16%  softirqs.CPU109.RCU
     11502 ± 13%     -41.5%       6726 ± 14%  softirqs.CPU11.RCU
      9003 ± 11%     -34.3%       5916 ± 12%  softirqs.CPU111.RCU
      9739 ±  7%     -36.5%       6187 ± 10%  softirqs.CPU113.RCU
      9453 ±  9%     -34.2%       6220 ± 13%  softirqs.CPU114.RCU
      9891 ±  7%     -31.3%       6798 ± 18%  softirqs.CPU115.RCU
     10008 ±  9%     -39.1%       6090 ± 12%  softirqs.CPU116.RCU
     10132 ± 17%     -41.7%       5912 ± 15%  softirqs.CPU117.RCU
      9765 ±  8%     -37.5%       6103 ±  8%  softirqs.CPU118.RCU
      8960 ± 11%     -33.9%       5923 ±  8%  softirqs.CPU119.RCU
     10910 ± 17%     -34.0%       7197 ±  6%  softirqs.CPU12.RCU
      9577 ±  6%     -37.6%       5973 ± 12%  softirqs.CPU120.RCU
      9565 ± 10%     -42.9%       5457 ±  7%  softirqs.CPU121.RCU
      9137 ± 10%     -38.7%       5604 ± 11%  softirqs.CPU124.RCU
      9418 ± 13%     -34.3%       6186 ± 13%  softirqs.CPU125.RCU
      9151 ± 15%     -37.2%       5745 ± 11%  softirqs.CPU126.RCU
     10800 ±  7%     -34.7%       7057 ± 10%  softirqs.CPU127.RCU
      9815 ±  9%     -29.1%       6958 ±  6%  softirqs.CPU13.RCU
     11463 ± 15%     -39.2%       6967 ± 12%  softirqs.CPU14.RCU
     10757 ± 15%     -37.6%       6716 ± 10%  softirqs.CPU15.RCU
     10809 ± 17%     -38.5%       6648 ± 16%  softirqs.CPU16.RCU
     11169 ± 14%     -42.4%       6435 ± 13%  softirqs.CPU17.RCU
     12215 ± 11%     -40.1%       7319 ± 13%  softirqs.CPU19.RCU
     11500 ± 12%     -39.2%       6996 ± 10%  softirqs.CPU2.RCU
     11027 ± 18%     -32.7%       7416 ±  9%  softirqs.CPU20.RCU
     11155 ± 17%     -39.5%       6745 ±  5%  softirqs.CPU21.RCU
     11083 ± 16%     -32.8%       7451 ± 12%  softirqs.CPU22.RCU
     11841 ± 15%     -44.9%       6523 ± 13%  softirqs.CPU23.RCU
     11415 ± 16%     -38.7%       6997 ± 15%  softirqs.CPU24.RCU
     11325 ±  5%     -37.7%       7061 ±  8%  softirqs.CPU25.RCU
     12111 ± 11%     -44.5%       6715 ± 11%  softirqs.CPU26.RCU
     11627 ± 11%     -43.1%       6613 ± 11%  softirqs.CPU27.RCU
     11113 ±  3%     -40.1%       6658 ± 10%  softirqs.CPU28.RCU
     10333 ± 13%     -33.2%       6907 ±  8%  softirqs.CPU29.RCU
     12191 ± 13%     -42.5%       7009 ± 13%  softirqs.CPU3.RCU
     10265 ± 11%     -31.7%       7012 ±  9%  softirqs.CPU30.RCU
     11440 ± 13%     -39.6%       6910 ± 17%  softirqs.CPU31.RCU
     10389 ± 12%     -33.8%       6874 ± 11%  softirqs.CPU32.RCU
     11224 ± 10%     -37.3%       7032 ± 19%  softirqs.CPU33.RCU
     10886 ± 13%     -32.8%       7317 ± 16%  softirqs.CPU34.RCU
     10588 ±  9%     -32.7%       7125 ±  6%  softirqs.CPU35.RCU
      9719 ± 16%     -31.8%       6625 ± 14%  softirqs.CPU36.RCU
     11099 ± 12%     -33.2%       7414 ± 12%  softirqs.CPU37.RCU
     10338 ±  9%     -25.6%       7690 ± 14%  softirqs.CPU38.RCU
     10238 ± 11%     -27.1%       7461 ±  9%  softirqs.CPU39.RCU
     11660 ± 15%     -35.5%       7516 ± 12%  softirqs.CPU4.RCU
     10552 ± 13%     -31.6%       7220 ± 12%  softirqs.CPU40.RCU
     10552 ± 10%     -31.1%       7273 ±  7%  softirqs.CPU41.RCU
     10103 ± 13%     -31.8%       6891 ±  9%  softirqs.CPU42.RCU
     10974 ±  8%     -36.9%       6921 ± 10%  softirqs.CPU43.RCU
     10730 ± 13%     -37.2%       6739 ± 17%  softirqs.CPU44.RCU
     11205 ± 12%     -35.4%       7233 ± 11%  softirqs.CPU45.RCU
     11136 ± 12%     -39.4%       6751 ± 12%  softirqs.CPU46.RCU
     10649 ± 11%     -33.6%       7073 ±  8%  softirqs.CPU47.RCU
      9988 ± 10%     -36.5%       6346 ±  8%  softirqs.CPU48.RCU
      9801 ±  9%     -36.9%       6180 ±  6%  softirqs.CPU49.RCU
     12102 ±  9%     -37.5%       7563 ± 17%  softirqs.CPU5.RCU
      9884 ± 12%     -36.2%       6303 ±  7%  softirqs.CPU50.RCU
      9347 ±  9%     -34.8%       6096 ±  7%  softirqs.CPU51.RCU
     10164 ± 11%     -35.3%       6575 ±  7%  softirqs.CPU52.RCU
     10518 ± 15%     -37.2%       6609 ± 10%  softirqs.CPU53.RCU
      9857 ± 12%     -32.5%       6655 ±  5%  softirqs.CPU54.RCU
     10795 ± 13%     -44.0%       6047 ±  5%  softirqs.CPU55.RCU
     10397 ± 12%     -38.9%       6350 ±  7%  softirqs.CPU56.RCU
      9513 ±  6%     -30.5%       6615 ±  8%  softirqs.CPU57.RCU
     10400 ± 10%     -37.6%       6492 ±  5%  softirqs.CPU58.RCU
     10434 ±  8%     -39.1%       6351 ±  9%  softirqs.CPU59.RCU
     12055 ± 14%     -32.8%       8102 ± 22%  softirqs.CPU6.RCU
     10580 ± 12%     -40.1%       6340 ± 11%  softirqs.CPU60.RCU
     10099 ± 11%     -41.4%       5915 ±  7%  softirqs.CPU61.RCU
     10021 ± 10%     -35.8%       6436 ±  3%  softirqs.CPU62.RCU
      9856 ±  7%     -31.3%       6774 ±  9%  softirqs.CPU64.RCU
      9963 ± 12%     -36.0%       6372 ±  8%  softirqs.CPU65.RCU
      9664 ±  8%     -30.1%       6752 ±  5%  softirqs.CPU66.RCU
      9673 ±  5%     -33.2%       6463 ± 10%  softirqs.CPU67.RCU
      9910 ±  8%     -36.4%       6307 ±  6%  softirqs.CPU68.RCU
      9589 ±  5%     -34.2%       6310 ±  7%  softirqs.CPU69.RCU
     11733 ±  9%     -42.0%       6809 ± 12%  softirqs.CPU7.RCU
      9603 ± 42%    +105.6%      19745 ± 32%  softirqs.CPU7.SCHED
      9881 ±  9%     -34.0%       6521 ±  7%  softirqs.CPU70.RCU
      9620 ±  6%     -33.2%       6424 ±  2%  softirqs.CPU71.RCU
     35921 ± 13%     -39.4%      21764 ± 30%  softirqs.CPU71.SCHED
     10659 ± 13%     -38.8%       6524 ±  7%  softirqs.CPU72.RCU
      9917 ±  7%     -38.9%       6055 ±  7%  softirqs.CPU73.RCU
      9572 ±  7%     -40.0%       5741 ±  9%  softirqs.CPU74.RCU
      9683 ±  9%     -36.2%       6176 ±  6%  softirqs.CPU75.RCU
     10099 ±  7%     -38.4%       6220 ± 10%  softirqs.CPU76.RCU
      9726 ±  5%     -28.6%       6946 ± 14%  softirqs.CPU78.RCU
      9463 ±  6%     -37.1%       5956 ± 12%  softirqs.CPU79.RCU
     10789 ± 12%     -34.1%       7105 ± 20%  softirqs.CPU8.RCU
      9917 ±  7%     -35.1%       6439 ± 11%  softirqs.CPU80.RCU
      9821 ±  4%     -34.5%       6430 ±  3%  softirqs.CPU81.RCU
      9938 ± 12%     -38.6%       6098 ± 10%  softirqs.CPU82.RCU
      9838 ±  4%     -35.2%       6370 ±  7%  softirqs.CPU83.RCU
     10401 ±  8%     -40.1%       6229 ±  6%  softirqs.CPU84.RCU
     10051 ±  6%     -36.4%       6396 ±  9%  softirqs.CPU85.RCU
      9792 ±  4%     -35.3%       6339 ± 14%  softirqs.CPU86.RCU
     10059 ±  9%     -36.3%       6408 ±  8%  softirqs.CPU87.RCU
      9972 ±  8%     -39.2%       6059 ±  3%  softirqs.CPU88.RCU
      9447 ±  9%     -30.8%       6534 ± 12%  softirqs.CPU89.RCU
     11511 ± 24%     -37.0%       7246 ± 11%  softirqs.CPU9.RCU
      9563 ±  6%     -30.9%       6612 ±  6%  softirqs.CPU90.RCU
     36244 ± 11%     -29.7%      25496 ± 25%  softirqs.CPU90.SCHED
      9671 ±  8%     -34.4%       6340 ±  7%  softirqs.CPU91.RCU
      9848 ±  7%     -33.7%       6531 ±  5%  softirqs.CPU92.RCU
      9578 ±  8%     -32.4%       6479 ±  5%  softirqs.CPU93.RCU
      9519 ±  5%     -36.3%       6064 ±  7%  softirqs.CPU94.RCU
      9480 ±  9%     -31.0%       6544 ± 12%  softirqs.CPU95.RCU
      9025 ±  8%     -35.2%       5845 ±  5%  softirqs.CPU96.RCU
      9656 ± 19%     -39.8%       5814 ±  8%  softirqs.CPU97.RCU
     10008 ±  7%     -39.4%       6064 ±  9%  softirqs.CPU98.RCU
      9619 ±  6%     -41.1%       5661 ±  3%  softirqs.CPU99.RCU
   1308899 ±  3%     -36.2%     835466 ±  3%  softirqs.RCU
     47568 ±  2%     +20.6%      57361 ±  4%  softirqs.TIMER
     10.02 ±  3%      -4.9        5.08 ± 18%  perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_fcntl64
      8.20            -3.8        4.38 ±  6%  perf-profile.calltrace.cycles-pp._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      5.28            -2.7        2.58 ±  5%  perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      4.78            -2.3        2.48 ±  7%  perf-profile.calltrace.cycles-pp.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      3.68            -1.8        1.88 ±  7%  perf-profile.calltrace.cycles-pp.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk.do_fcntl
      3.36            -1.4        1.93 ±  6%  perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64
      2.98            -1.2        1.75 ±  6%  perf-profile.calltrace.cycles-pp.__fget_light.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      2.51            -1.2        1.30 ±  7%  perf-profile.calltrace.cycles-pp.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      2.49            -1.1        1.36 ±  6%  perf-profile.calltrace.cycles-pp._raw_spin_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      2.02            -1.0        1.05 ±  6%  perf-profile.calltrace.cycles-pp.locks_unlink_lock_ctx.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      2.02            -1.0        1.06 ±  7%  perf-profile.calltrace.cycles-pp.locks_insert_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      1.14            -0.8        0.37 ± 70%  perf-profile.calltrace.cycles-pp.locks_release_private.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      1.70            -0.7        0.98 ±  7%  perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64
      1.24            -0.6        0.64 ±  7%  perf-profile.calltrace.cycles-pp._raw_spin_lock.locks_insert_lock_ctx.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      2.15            -0.6        1.57 ±  8%  perf-profile.calltrace.cycles-pp.memset_erms.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      1.34            -0.6        0.77 ±  6%  perf-profile.calltrace.cycles-pp.___might_sleep.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      1.32            -0.6        0.75 ±  7%  perf-profile.calltrace.cycles-pp.__might_fault._copy_from_user.do_fcntl.__x64_sys_fcntl.do_syscall_64
      1.22            -0.5        0.68 ±  6%  perf-profile.calltrace.cycles-pp._raw_spin_lock.locks_unlink_lock_ctx.locks_delete_lock_ctx.posix_lock_inode.do_lock_file_wait
      1.21            -0.5        0.68 ±  7%  perf-profile.calltrace.cycles-pp.aa_file_perm.common_file_perm.security_file_lock.do_lock_file_wait.fcntl_setlk
      1.41            -0.5        0.90 ±  9%  perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      1.21            -0.5        0.70 ±  6%  perf-profile.calltrace.cycles-pp.syscall_enter_from_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      1.08            -0.5        0.61 ±  7%  perf-profile.calltrace.cycles-pp.flock64_to_posix_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
      0.76            -0.4        0.35 ± 70%  perf-profile.calltrace.cycles-pp.___might_sleep.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      1.27            -0.4        0.86 ±  5%  perf-profile.calltrace.cycles-pp.__might_sleep.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      1.04            -0.4        0.66 ± 10%  perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_fcntl64
      0.76            -0.1        0.63 ±  5%  perf-profile.calltrace.cycles-pp._raw_spin_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
     97.66            +0.4       98.07        perf-profile.calltrace.cycles-pp.__libc_fcntl64
      0.00            +0.5        0.54 ±  4%  perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      0.00            +0.8        0.83 ±  7%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      0.00            +0.9        0.90 ±  7%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      0.00            +0.9        0.92 ±  5%  perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      0.00            +1.3        1.26 ±  7%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      2.15            +1.4        3.54 ± 14%  perf-profile.calltrace.cycles-pp.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      0.00            +1.6        1.59 ± 11%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      1.14            +1.8        2.90 ± 18%  perf-profile.calltrace.cycles-pp.kmem_cache_free.locks_dispose_list.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      0.00            +1.8        1.83 ±  6%  perf-profile.calltrace.cycles-pp.mod_objcg_state.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      0.00            +2.1        2.13 ± 29%  perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc
      0.00            +2.2        2.16 ± 29%  perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk
      0.00            +2.4        2.42 ±  7%  perf-profile.calltrace.cycles-pp.get_obj_cgroup_from_current.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
      0.00            +2.6        2.60 ± 28%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl
      0.00            +2.7        2.66 ± 27%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl
      0.00            +3.0        3.04 ± 37%  perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode
      2.86 ±  2%      +3.6        6.42 ± 10%  perf-profile.calltrace.cycles-pp.kmem_cache_free.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
      0.00            +3.6        3.59 ± 33%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk
      0.00            +3.6        3.65 ± 36%  perf-profile.calltrace.cycles-pp.page_counter_uncharge.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait
      0.00            +3.7        3.72 ± 36%  perf-profile.calltrace.cycles-pp.obj_cgroup_uncharge_pages.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk
      0.00            +4.3        4.29 ± 27%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl
      3.76            +5.7        9.47 ± 12%  perf-profile.calltrace.cycles-pp.kmem_cache_free.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
      8.70            +6.2       14.88 ±  6%  perf-profile.calltrace.cycles-pp.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
      7.40            +6.3       13.74 ±  7%  perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.fcntl_setlk.do_fcntl.__x64_sys_fcntl
     83.98            +6.8       90.74        perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
     82.41            +7.4       89.82        perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      0.00            +7.8        7.75 ± 29%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode
     46.22            +7.9       54.10 ±  3%  perf-profile.calltrace.cycles-pp.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64
     78.68            +8.9       87.56        perf-profile.calltrace.cycles-pp.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      0.00            +9.2        9.15 ± 23%  perf-profile.calltrace.cycles-pp.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait
     73.64           +11.0       84.60        perf-profile.calltrace.cycles-pp.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fcntl64
      0.00           +11.2       11.21 ± 25%  perf-profile.calltrace.cycles-pp.page_counter_try_charge.obj_cgroup_charge_pages.obj_cgroup_charge.kmem_cache_alloc.locks_alloc_lock
     38.04           +11.7       49.70 ±  4%  perf-profile.calltrace.cycles-pp.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl.__x64_sys_fcntl
     16.21           +11.8       28.04 ±  5%  perf-profile.calltrace.cycles-pp.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk.do_fcntl
     13.60           +12.1       25.71 ±  6%  perf-profile.calltrace.cycles-pp.kmem_cache_alloc.locks_alloc_lock.posix_lock_inode.do_lock_file_wait.fcntl_setlk
     63.80           +15.2       78.96        perf-profile.calltrace.cycles-pp.fcntl_setlk.do_fcntl.__x64_sys_fcntl.do_syscall_64.entry_SYSCALL_64_after_hwframe
      7.76            -3.4        4.33 ±  6%  perf-profile.children.cycles-pp.memset_erms
      8.05            -3.4        4.62 ±  7%  perf-profile.children.cycles-pp._copy_from_user
      5.70 ±  2%      -2.8        2.89 ± 18%  perf-profile.children.cycles-pp.__entry_text_start
      6.00            -2.5        3.47 ±  6%  perf-profile.children.cycles-pp._raw_spin_lock
      5.10            -2.4        2.66 ±  6%  perf-profile.children.cycles-pp.security_file_lock
      5.00            -2.2        2.75 ±  8%  perf-profile.children.cycles-pp.syscall_return_via_sysret
      3.90            -1.9        2.01 ±  7%  perf-profile.children.cycles-pp.common_file_perm
      3.94            -1.7        2.28 ±  6%  perf-profile.children.cycles-pp.copy_user_generic_unrolled
      3.74            -1.7        2.08 ±  5%  perf-profile.children.cycles-pp.___might_sleep
      3.95            -1.6        2.40 ±  6%  perf-profile.children.cycles-pp.__might_sleep
      2.72            -1.3        1.41 ±  6%  perf-profile.children.cycles-pp.locks_delete_lock_ctx
      2.58            -1.3        1.29 ±  5%  perf-profile.children.cycles-pp.locks_release_private
      3.10            -1.3        1.81 ±  6%  perf-profile.children.cycles-pp.__fget_light
      2.18            -1.0        1.14 ±  6%  perf-profile.children.cycles-pp.locks_unlink_lock_ctx
      2.18            -1.0        1.15 ±  7%  perf-profile.children.cycles-pp.locks_insert_lock_ctx
      2.03            -0.9        1.17 ±  7%  perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
      2.19            -0.8        1.34 ±  5%  perf-profile.children.cycles-pp.__cond_resched
      1.54            -0.7        0.86 ±  7%  perf-profile.children.cycles-pp.aa_file_perm
      1.54            -0.7        0.87 ±  7%  perf-profile.children.cycles-pp.__might_fault
      1.68            -0.6        1.07 ±  9%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode
      1.32            -0.6        0.76 ±  6%  perf-profile.children.cycles-pp.syscall_enter_from_user_mode
      1.19            -0.5        0.67 ±  6%  perf-profile.children.cycles-pp.flock64_to_posix_lock
      1.10            -0.4        0.65 ±  5%  perf-profile.children.cycles-pp.rcu_all_qs
      0.81            -0.4        0.45 ±  7%  perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
      0.94            -0.3        0.60 ±  6%  perf-profile.children.cycles-pp.__init_waitqueue_head
      0.74            -0.3        0.41 ±  4%  perf-profile.children.cycles-pp.locks_delete_block
      0.75            -0.3        0.42 ±  5%  perf-profile.children.cycles-pp.locks_get_lock_context
      0.70            -0.3        0.39 ±  7%  perf-profile.children.cycles-pp.locks_copy_lock
      0.53            -0.3        0.23 ±  5%  perf-profile.children.cycles-pp.__list_del_entry_valid
      0.78 ±  2%      -0.3        0.49 ±  9%  perf-profile.children.cycles-pp.exit_to_user_mode_prepare
      0.61            -0.3        0.35 ±  8%  perf-profile.children.cycles-pp.security_file_fcntl
      0.60            -0.2        0.35 ± 11%  perf-profile.children.cycles-pp.memset
      0.53            -0.2        0.29 ±  6%  perf-profile.children.cycles-pp.apparmor_file_lock
      0.43 ±  2%      -0.2        0.24 ±  7%  perf-profile.children.cycles-pp.__list_add_valid
      0.43 ±  2%      -0.2        0.25 ±  7%  perf-profile.children.cycles-pp.vfs_lock_file
      0.50            -0.2        0.33 ±  8%  perf-profile.children.cycles-pp.testcase
      0.37            -0.2        0.21 ±  6%  perf-profile.children.cycles-pp.locks_copy_conflock
      0.32 ±  2%      -0.1        0.18 ±  7%  perf-profile.children.cycles-pp.__fdget_raw
      0.36            -0.1        0.24 ±  7%  perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
      0.22            -0.1        0.12 ±  6%  perf-profile.children.cycles-pp.locks_move_blocks
      0.41            -0.1        0.32 ±  6%  perf-profile.children.cycles-pp.should_failslab
      0.20 ±  2%      -0.1        0.13 ± 10%  perf-profile.children.cycles-pp.fcntl@plt
      0.00            +0.1        0.08 ±  6%  perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg
      0.00            +0.1        0.08 ± 11%  perf-profile.children.cycles-pp.refill_stock
     98.10            +0.1       98.23        perf-profile.children.cycles-pp.__libc_fcntl64
      0.00            +0.3        0.30 ±  6%  perf-profile.children.cycles-pp.obj_cgroup_uncharge
      0.00            +0.5        0.53 ±  7%  perf-profile.children.cycles-pp.mem_cgroup_from_task
      2.41            +1.3        3.69 ± 14%  perf-profile.children.cycles-pp.locks_dispose_list
      0.00            +2.0        1.99 ±  5%  perf-profile.children.cycles-pp.refill_obj_stock
      0.32 ±  2%      +2.1        2.40 ±  6%  perf-profile.children.cycles-pp.rcu_read_unlock_strict
      0.00            +3.4        3.39 ± 21%  perf-profile.children.cycles-pp.propagate_protected_usage
      0.00            +4.9        4.90 ±  8%  perf-profile.children.cycles-pp.get_obj_cgroup_from_current
      0.00            +5.6        5.60 ±  6%  perf-profile.children.cycles-pp.mod_objcg_state
      0.00            +5.9        5.94 ± 26%  perf-profile.children.cycles-pp.page_counter_cancel
     84.29            +6.7       90.98        perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
      0.00            +7.1        7.12 ± 24%  perf-profile.children.cycles-pp.page_counter_uncharge
     83.10            +7.2       90.30        perf-profile.children.cycles-pp.do_syscall_64
      0.00            +7.3        7.27 ± 24%  perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
     46.87            +7.6       54.45 ±  3%  perf-profile.children.cycles-pp.do_lock_file_wait
     79.26            +8.6       87.90        perf-profile.children.cycles-pp.__x64_sys_fcntl
     74.02           +10.8       84.84        perf-profile.children.cycles-pp.do_fcntl
     39.59           +11.0       50.57 ±  4%  perf-profile.children.cycles-pp.posix_lock_inode
      0.00           +11.2       11.23 ± 25%  perf-profile.children.cycles-pp.page_counter_try_charge
      0.00           +11.4       11.37 ± 24%  perf-profile.children.cycles-pp.obj_cgroup_charge_pages
      8.04           +11.5       19.54 ±  6%  perf-profile.children.cycles-pp.kmem_cache_free
      0.00           +13.6       13.63 ± 19%  perf-profile.children.cycles-pp.obj_cgroup_charge
     64.58           +14.8       79.39        perf-profile.children.cycles-pp.fcntl_setlk
     25.73           +17.7       43.45 ±  2%  perf-profile.children.cycles-pp.locks_alloc_lock
     22.71           +18.8       41.55 ±  3%  perf-profile.children.cycles-pp.kmem_cache_alloc
      7.42            -3.3        4.14 ±  6%  perf-profile.self.cycles-pp.memset_erms
      6.55            -3.2        3.35 ±  8%  perf-profile.self.cycles-pp.__libc_fcntl64
      5.72            -2.4        3.30 ±  6%  perf-profile.self.cycles-pp._raw_spin_lock
      5.14            -2.4        2.76 ±  5%  perf-profile.self.cycles-pp.posix_lock_inode
      5.00            -2.2        2.75 ±  8%  perf-profile.self.cycles-pp.syscall_return_via_sysret
      3.49            -1.7        1.77 ±  4%  perf-profile.self.cycles-pp.fcntl_setlk
      3.61            -1.5        2.09 ±  6%  perf-profile.self.cycles-pp.copy_user_generic_unrolled
      3.20            -1.4        1.77 ±  5%  perf-profile.self.cycles-pp.___might_sleep
      2.97            -1.2        1.74 ±  6%  perf-profile.self.cycles-pp.__fget_light
      2.35            -1.2        1.14 ±  7%  perf-profile.self.cycles-pp.common_file_perm
      3.02            -1.2        1.84 ±  6%  perf-profile.self.cycles-pp.__might_sleep
      2.25            -1.1        1.14 ±  5%  perf-profile.self.cycles-pp.locks_release_private
      2.38            -0.9        1.47 ±  8%  perf-profile.self.cycles-pp.locks_alloc_lock
      1.80            -0.8        1.04 ±  7%  perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
      1.39            -0.7        0.71 ± 21%  perf-profile.self.cycles-pp.__entry_text_start
      1.61            -0.7        0.94 ±  7%  perf-profile.self.cycles-pp.__x64_sys_fcntl
      1.60            -0.6        0.95 ±  8%  perf-profile.self.cycles-pp.do_fcntl
      1.43            -0.6        0.79 ±  6%  perf-profile.self.cycles-pp.do_lock_file_wait
      1.40            -0.6        0.81 ±  7%  perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
      1.31            -0.6        0.73 ±  7%  perf-profile.self.cycles-pp.aa_file_perm
      1.16            -0.5        0.67 ±  6%  perf-profile.self.cycles-pp.syscall_enter_from_user_mode
      1.08            -0.5        0.60 ±  6%  perf-profile.self.cycles-pp.flock64_to_posix_lock
      0.97            -0.4        0.56 ±  6%  perf-profile.self.cycles-pp._copy_from_user
      1.02            -0.4        0.61 ±  8%  perf-profile.self.cycles-pp.do_syscall_64
      1.04            -0.4        0.65 ±  4%  perf-profile.self.cycles-pp.__cond_resched
      0.81            -0.4        0.45 ±  7%  perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
      0.76            -0.4        0.42 ±  5%  perf-profile.self.cycles-pp.security_file_lock
      0.71            -0.3        0.38 ±  8%  perf-profile.self.cycles-pp.locks_insert_lock_ctx
      0.64            -0.3        0.32 ±  6%  perf-profile.self.cycles-pp.locks_unlink_lock_ctx
      0.75 ±  2%      -0.3        0.44 ±  6%  perf-profile.self.cycles-pp.rcu_all_qs
      0.70            -0.3        0.42 ± 10%  perf-profile.self.cycles-pp.exit_to_user_mode_prepare
      0.58 ±  2%      -0.3        0.32 ±  5%  perf-profile.self.cycles-pp.locks_dispose_list
      0.43            -0.3        0.18 ±  7%  perf-profile.self.cycles-pp.__list_del_entry_valid
      0.52            -0.2        0.29 ±  5%  perf-profile.self.cycles-pp.locks_delete_block
      0.53            -0.2        0.30 ±  5%  perf-profile.self.cycles-pp.locks_get_lock_context
      0.63 ±  2%      -0.2        0.42 ±  5%  perf-profile.self.cycles-pp.__init_waitqueue_head
      0.43            -0.2        0.23 ±  9%  perf-profile.self.cycles-pp.locks_delete_lock_ctx
      0.56 ±  2%      -0.2        0.36 ± 11%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode
      0.43            -0.2        0.25 ±  9%  perf-profile.self.cycles-pp.security_file_fcntl
      0.32 ±  2%      -0.2        0.17 ±  7%  perf-profile.self.cycles-pp.__list_add_valid
      0.33            -0.2        0.18 ±  9%  perf-profile.self.cycles-pp.locks_copy_lock
      0.33            -0.1        0.18 ±  7%  perf-profile.self.cycles-pp.__might_fault
      0.42 ±  2%      -0.1        0.27 ±  9%  perf-profile.self.cycles-pp.testcase
      0.33 ±  2%      -0.1        0.18 ±  5%  perf-profile.self.cycles-pp.vfs_lock_file
      0.32            -0.1        0.18 ±  7%  perf-profile.self.cycles-pp.apparmor_file_lock
      0.32 ±  2%      -0.1        0.18 ±  6%  perf-profile.self.cycles-pp.locks_copy_conflock
      0.30 ±  3%      -0.1        0.18 ± 10%  perf-profile.self.cycles-pp.memset
      0.26 ±  3%      -0.1        0.18 ±  7%  perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
      0.17 ±  2%      -0.1        0.09 ±  7%  perf-profile.self.cycles-pp.locks_move_blocks
      0.11 ±  4%      -0.0        0.06 ±  6%  perf-profile.self.cycles-pp.__fdget_raw
      0.20 ±  3%      -0.0        0.17 ±  6%  perf-profile.self.cycles-pp.should_failslab
      0.10 ±  4%      -0.0        0.07 ± 14%  perf-profile.self.cycles-pp.fcntl@plt
      0.00            +0.1        0.06 ±  6%  perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg
      0.00            +0.1        0.08 ± 11%  perf-profile.self.cycles-pp.refill_stock
      0.00            +0.1        0.14 ± 10%  perf-profile.self.cycles-pp.obj_cgroup_uncharge
      0.00            +0.4        0.36 ±  7%  perf-profile.self.cycles-pp.mem_cgroup_from_task
      0.11 ±  3%      +0.8        0.92 ±  5%  perf-profile.self.cycles-pp.rcu_read_unlock_strict
      0.00            +1.8        1.85 ±  5%  perf-profile.self.cycles-pp.refill_obj_stock
      0.00            +2.0        2.05 ±  7%  perf-profile.self.cycles-pp.obj_cgroup_charge
      9.35            +3.0       12.32 ±  4%  perf-profile.self.cycles-pp.kmem_cache_alloc
      0.00            +3.4        3.36 ± 21%  perf-profile.self.cycles-pp.propagate_protected_usage
      0.00            +4.0        4.01 ±  8%  perf-profile.self.cycles-pp.get_obj_cgroup_from_current
      0.00            +5.2        5.23 ±  7%  perf-profile.self.cycles-pp.mod_objcg_state
      0.00            +5.9        5.90 ± 26%  perf-profile.self.cycles-pp.page_counter_cancel
      0.00            +9.0        8.97 ± 25%  perf-profile.self.cycles-pp.page_counter_try_charge





Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


---
0DAY/LKP+ Test Infrastructure                   Open Source Technology Center
https://lists.01.org/hyperkitty/list/lkp@lists.01.org       Intel Corporation

Thanks,
Oliver Sang


View attachment "config-5.14.0-00066-g0f12156dff28" of type "text/plain" (175510 bytes)

View attachment "job-script" of type "text/plain" (7669 bytes)

View attachment "job.yaml" of type "text/plain" (5206 bytes)

View attachment "reproduce" of type "text/plain" (337 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ