lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:	Sat, 7 Sep 2013 20:37:28 +0800
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Yuanhan Liu <yuanhan.liu@...ux.intel.com>,
	"Huang, Ying" <ying.huang@...el.com>,
	Mike Galbraith <efault@....de>, Ingo Molnar <mingo@...nel.org>,
	LKML <linux-kernel@...r.kernel.org>, lkp@...org
Subject: Re: 3-5% increased netperf throughput by "sched: Micro-optimize the
 smart wake-affine logic"

On Sat, Sep 07, 2013 at 08:32:51PM +0800, Yuanhan Liu wrote:
> Hi Peter,
> 
> We are glad to report some measurable performance improvements by your
> commit
> 
> commit 7d9ffa8961482232d964173cccba6e14d2d543b2
> Author: Peter Zijlstra <peterz@...radead.org>
> Date:   Thu Jul 4 12:56:46 2013 +0800
> 
>     sched: Micro-optimize the smart wake-affine logic

FYI, here is the direct comparison with its parent commit.


                              netperf.Throughput_Mbps

   252 ++---O-----------------------O-----------------------------O---------+
       | O    O    O                         O      O           O    O      O
   250 O+        O        O O  O O    O O  O      O    O               O    |
       |             O                         O         O    O           O |
       |                                                                    |
   248 ++               O                                  O                |
       |                                                                    |
   246 ++                                                                   |
       |                                                                    |
   244 ++        *                                                          |
       *        : :         *..     *   *         *    *          *..     * |
       |+       : :        +      .. :  :+      .. :  + :        :      ..  |
   242 ++*..*  :   :   .*.*    *.*   : :  + .*.*   : +  : .*..*. :   *.*    |
       |     + :   *.*.               ::   *        *    *      *           |
   240 ++-----*-----------------------*-------------------------------------+



                              netperf.Throughput_Mbps

   208 ++-------------------------------------------------------------------+
       |                                O                                   |
   206 ++   O      O O      O    O    O                O        O         O |
   204 O+     O  O      O O    O           O   O  O O    O    O        O    |
       | O                          O        O             O      O  O      O
   202 ++                                                                   |
   200 ++                                                                   |
       |                                                                    |
   198 ++                                                                   |
   196 ++                                                                   |
       |                                                     .*             |
   194 ++*..*.  .*.     *. .*..*.*..          .*.. .*..*. .*.  + .*..*.  .*.|
   192 *+     *.   *. ..  *         *.*.*..*.*    *      *      *      *.   *
       |             *                                                      |
   190 ++-------------------------------------------------------------------+



                              netperf.Throughput_Mbps

   250 ++-----------------------------------------O-------------------------+
       |                       O      O O           O      O                |
   248 ++O    O  O      O   O       O          O         O        O  O      O
       O    O      O O           O                            O O      O  O |
   246 ++                 O                O O                              |
       |                                               O                    |
   244 ++                                                                   |
       |                                                                    |
   242 ++                                                            *      *
       |                                                            : :    +|
   240 ++                                                           : :   * |
       *.  .*.*.. .*     .*         *   *..*.     *.  .*.*.*..*.   :   :..  |
   238 ++*.      *  +  .*  +  .*. .. + +     *   +  *.          *.*    *    |
       |             *.     *.   *    *       + +                           |
   236 ++--------------------------------------*----------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  242.00        +3.4%       250.17  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                  193.33        +5.7%       204.30  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                  238.57        +3.8%       247.63  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                  673.90        +4.2%       702.10  TOTAL netperf.Throughput_Mbps


                                hackbench.throughput

   37900 ++-----------------------------------------------------------------+
         |                                    O                             |
   37800 ++     O                                     O                     |
   37700 ++                                                                 O
         O              O      O       O                                    |
   37600 ++     *.                                                          |
   37500 ++    :  .            *...                          O       O      |
         |    :    .          .    ...              ..*......*.......    ...*
   37400 ++  :      ..       .        .         ....                 *...   |
   37300 ++  :        .    ..          *......*.                            |
         |  :          .  .                                                 |
   37200 ++:             .                                                  |
   37100 ++             *                                                   |
         *                                                                  |
   37000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                37361.57        +0.8%     37661.84  lkp-snb01/micro/hackbench/1600%-threads-pipe
                37361.57        +0.8%     37661.84  TOTAL hackbench.throughput


                                 interrupts.CPU0.RES

   30000 ++-----------------------------------------------------------------+
         |                                                    O             |
   25000 ++                                                                 |
         |                                                                  |
         |                                                                  |
   20000 ++                            O                                    |
         |           O   O           O   O                                  |
   15000 O+                     O O           O    O            O  O        *
         |    O O          O                    O    O O  O                :|
   10000 *+O                    *           O   *..         O          O   :|
         |+  .*.  O    O.*.*..O+ +       *..   +          *.*. .*..  O    * |
         | *.   *.*..*.*      *   *..*. +   *.*    *.   ..    *    *.*. ..  |
    5000 ++                            *             *.*               *    |
         |                                                                  |
       0 ++-----------------------------------------------------------------+



                                  interrupts.CPU0.RES

   1.1e+06 ++---------------------------------------------------------------+
           | O      O                                 O            O        |
     1e+06 ++                                  O                            |
    900000 ++              O                                                |
           |                     O      O                               O   |
    800000 ++                                                               |
    700000 ++         O                          O          O    *          |
           |                          O   O      *               :   O  *   |
    600000 O+  *..*     *..                  O   :+  .*         :O:     :   |
    500000 ++ +   O:   :     O      O           :  *.  +O O    O: :    : :O O
           | *     :   :   *.* O *              :       *. .*.. : :    : :  |
    400000 ++       *.*       : : :   *.*.*..   :         *    :  :   :   : |
    300000 *+                 : :  : :       *.:               *   :  :   *.*
           |                   *   : :         *                   *.:      |
    200000 ++-----------------------*--------------------------------*------+



                                  interrupts.CPU0.RES

   350000 ++--------------------------------O-------------------------------+
          |      O               O                                          |
   300000 ++       O                          O           O                 |
          |                                          O        O    O   O    O
          | O        O                           O          O    O        O |
   250000 O+            O O O O    O    O          O                        |
          |    O                     O                  O            O      |
   200000 ++                              O                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                 *                               |
          |.*.. .*   *..  *.*.*.. .*.      + +           .*.*     .*. .*..*.|
   100000 *+   *  + +    +       *   *..*.*   *..*.*.*..*    +  .*   *      *
          |        *    *                                     *.            |
    50000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 7475.21       +84.0%     13755.06  kbuildx/micro/netperf/120s-200%-TCP_STREAM
               427012.48       +62.8%    695138.90  vpx/micro/hackbench/1600%-process-pipe
                99524.61      +166.9%    265617.80  vpx/micro/hackbench/1600%-threads-socket
               534012.30       +82.5%    974511.76  TOTAL interrupts.CPU0.RES


                                 interrupts.CPU2.RES

   26000 ++-----------------------------------------------------------------+
   24000 ++          O                                                      |
         |               O                                      O           |
   22000 ++                            O                                    |
   20000 ++                                                   O             |
   18000 ++                                                                 |
   16000 O+     O                                           O               |
         |                                           O             O        *
   14000 ++       O             *           O      O   O             O      |
   12000 ++O                    O O  O        O           O                :|
   10000 ++                *.. : :       O      *                          :|
    8000 ++            O  :O   : :             : :                     O  : |
         *.  .*.          :   *   :        .*. :O :      .*.   .*..*.    .* |
    6000 ++*.   *.*..*.*.*        *..*.*.*.   *   : .*.*.   *.*      *.*.   |
    4000 ++----------------------------------------*------------------------+



                                 interrupts.CPU2.RES

   14500 ++----------------------------------------------------------O------+
         |                                                                  |
   14000 ++                O  O                                             |
   13500 ++                                                                 |
         |             O             O             O      O        O   O    O
   13000 O+       O                    O    O                 O *           |
         |      O    O   O      O O           O      O      O   O:          *
   12500 ++   O   *      *    *          O                     : :   *    O |
         *    *  : +    : :  + :                       O       :  :  :+    :|
   12000 ++  : + :  +   : : +  :  *..*.     *.*                :  : :  *.. :|
   11500 ++  :  *    *.*   *    :+     *   +   +O      *..    :    ::     * |
         | O:                   *       + +     *..*. +   *   *    *        |
   11000 ++*                             *           *     + +              |
         |                                                  *               |
   10500 ++-----------------------------------------------------------------+



                                  interrupts.CPU2.RES

   350000 ++----------------------------------------------------------------+
          |                                                 O               |
   300000 ++     O            O  O          O O                             |
          |                                      O            O        O  O |
          | O        O             O                 O    O                 |
   250000 O+       O        O                           O          O O      O
          |             O O          O             O             O          |
   200000 ++   O                        O O                                 |
          |                                                                 |
   150000 ++                                                                |
          |                                      *.       *            *..  |
          *.     *   *..                 .*.   ..  *.    + + .*..     :   *.|
   100000 ++*.. + + +    .*. .*..*.*.  .*   *.*      *..*   *    *.*. :     *
          |    *   *    *   *        *.                              *      |
    50000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 6873.32       +99.1%     13682.81  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                11794.09        +9.9%     12960.61  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                97821.37      +164.3%    258556.35  vpx/micro/hackbench/1600%-threads-socket
               116488.77      +144.8%    285199.77  TOTAL interrupts.CPU2.RES


                                 interrupts.CPU3.RES

   24000 ++-----------------------------------------------------------------+
   22000 ++                            O                      O             |
         |                                                                  |
   20000 ++              O                                                  |
   18000 ++          O            O                                         |
   16000 ++                                                     O           |
   14000 O+   O            *                                                |
         | O    O          :    *        O  O O    O O O  O        O        |
   12000 *+               : :   :                           O               *
   10000 ++            O  :O:  : :   O                                 O   :|
    8000 ++       O      :   : : :                                        O:|
    6000 ++: .*.        .*   :O:  :     .*..    O         *.*. .*..  O   .* |
         | *.   *.*..*.*      :   *..*.*    *.*.  .*.*. ..    *    *.*.*.   |
    4000 ++                   *                 *.     *                    |
    2000 ++-----------------------------------------------------------------+



                                 interrupts.CPU3.RES

   70000 ++-----------------------------------------------------------------+
         |                                    *.      O                     |
   65000 ++                                  :  ..                 ..*......*
   60000 ++                                 :     .            ....         |
         *..                ...*.      O    :      .        .*.             |
   55000 ++ ..         .*...     ..        :        ..    ..                |
         |    ..    ...            .      :             ..                  |
   50000 O+       ..                .    :            *.                    O
         |      *.      O            ..  :                                  |
   45000 ++                             :                    O              |
   40000 ++                            *      O                      O      |
         |                                                                  |
   35000 ++                                                                 |
         |                     O                                            |
   30000 ++-----------------------------------------------------------------+



                                  interrupts.CPU3.RES

   1.2e+06 ++---------------------------------------------------------------+
   1.1e+06 ++                           O                                   |
           |                                   O                            |
     1e+06 ++O             O     O                          O               |
    900000 ++       O                                                       |
           |                                          O            O    O   |
    800000 ++                         O                                     |
    700000 ++                  O    O              O             O          O
    600000 O+  O .*     *        *        O  O   O    *.* O    O *   O      |
           |  .*.  :    :+   O  : +              *   +   :       ::       O |
    500000 ++*     :  O:  + .*. :  +             :+ +    :      : :    .*.*.|
    400000 *+       :  :   *   *    *. .*.  .*  :  *      :     :  :.*.     *
           |        *.*               *   *.  + :         *.*..*   *        |
    300000 ++                                  *                            |
    200000 ++---------------------------------------------------------------+



                                  interrupts.CPU3.RES

   400000 ++----------------------------------------------------------------+
          |                                                            O    |
   350000 ++                                                                |
          |      O                                          O               |
   300000 ++                     O          O O               O             |
          | O      O O                               O    O      O O      O O
   250000 O+                O      O             O                   O      |
          |    O        O O   O      O  O O        O    O                   |
   200000 ++                                                                |
          |                                                                 |
   150000 ++         *                      *                               |
          |     .*  : +                    + +       *.. .*.           *..  |
   100000 *+*..*  + :  +   .*.*.. .*.*..*.*   *..*. +   *   *.    .*  :   *.*
          |        *    *.*      *                 *          *..*  + :     |
    50000 ++---------------------------------------------------------*------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 6424.37      +101.8%     12966.71  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                56672.74       -15.5%     47879.07  lkp-snb01/micro/hackbench/1600%-threads-socket
               446526.64       +66.6%    743914.14  vpx/micro/hackbench/1600%-process-pipe
                99614.97      +167.8%    266775.39  vpx/micro/hackbench/1600%-threads-socket
               609238.72       +75.9%   1071535.31  TOTAL interrupts.CPU3.RES


                                 interrupts.CPU1.RES

   24000 ++-----------------------------------------------------------------+
   22000 ++                                                   O             |
         |                             O                                    |
   20000 ++          O                                      O   O           |
   18000 ++              O                                         O        |
         |                                                                  |
   16000 O+     O                        O  O        O                      |
   14000 *+                          O        O        O                    *
   12000 ++O  O   O    O        *                  O                       :|
         |:                O    : O            .*         O          O O   :|
   10000 ++                *  O: :            *  :        *.              * |
    8000 ++:   .*.        + +  : :       *.. +    :      +  *.*.        ..O |
         | *..*   *..   .*   +:   :     +   *     :     +       *..   .*    |
    6000 ++          *.*      *   *..*.*           *.*.*           *.*      |
    4000 ++-----------------------------------------------------------------+



                                 interrupts.CPU1.RES

   80000 ++-----------------------------------------------------------------+
         |                                                           *......|
   75000 ++                                                        ..       *
   70000 ++                                                       .         |
         *..                                                     .          |
   65000 ++ ..                .*.                              ..           |
   60000 ++   ..          ....   ..                   O       .             |
         |          ....*.         .   O                  ...*              |
   55000 O+     *...                .        .*.......*...                  |
   50000 ++     O                    ..    ..                               O
         |              O                ..   O              O              |
   45000 ++                            *.                            O      |
   40000 ++                                                                 |
         |                     O                                            |
   35000 ++-----------------------------------------------------------------+



                                  interrupts.CPU1.RES

   1.1e+06 ++---------------------------------------------------------------+
           |                     O                                          |
     1e+06 ++O                                 O                   O        |
    900000 ++       O                                 O                     |
           |               O                                O               |
    800000 ++                           O                               O   |
    700000 ++           *                                                   |
           |            :                                 O                 |
    600000 ++  O .*    : :                         O    O        *          O
    500000 O+  *. O:   : :   O O *    O          *               O        O |
           |   :   :   :O :     : : O        O   O:   *.        : :  O  *   |
    400000 *+ :     *.O   :     :  :  *.  O     : : ..  *.     O: :   .. :  |
    300000 ++ :       *    :.*.*   : +  *.  .*  :  *      *.*..*   :.*   : .*
           | *             *        *     *.  +:                   *      * |
    200000 ++----------------------------------*----------------------------+



                                  interrupts.CPU1.RES

   400000 ++----------------------------------------------------------------+
          |                                                                 |
   350000 ++                                O                               |
          |      O               O                                     O    |
   300000 ++       O          O               O           O O               |
          O O        O    O                                   O           O |
   250000 ++            O          O                 O  O          O        O
          |                 O        O  O O      O               O          |
   200000 ++                                                                |
          |    O                                   O                 O      |
   150000 ++                                                                |
          *.    .*.*.         *..          .*            .*.*          *..  |
   100000 ++*..*     *.. .*. +      .*.. .*  +  .*.*.  .*    +        +   *.|
          |             *   *    *.*    *     *.     *.       *..*.*.*      *
    50000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 7767.42       +87.0%     14523.11  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                60441.90       -16.8%     50268.77  lkp-snb01/micro/hackbench/1600%-threads-socket
               376070.19       +67.7%    630658.76  vpx/micro/hackbench/1600%-process-pipe
                96980.98      +167.9%    259817.07  vpx/micro/hackbench/1600%-threads-socket
               541260.49       +76.5%    955267.71  TOTAL interrupts.CPU1.RES


                                   interrupts.RES

   24000 ++-----------------------------------------------------------------+
         |                                                             O    |
   22000 ++                                                                 |
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   18000 O+              O                      O  O                        |
         |      *                                    O                    O |
   16000 ++     ::   O     O             O    O           *   O             |
         |     : :O           O      O O               O : :    O           O
   14000 ++O  O:O :   .*        O O         O.*.         : :O      O O    * |
         *.   :   *..*  +    .*.*.  .*.*.  .*   *..     :   :.*.        .. +|
   12000 ++*..*          *.*.     *.     *.        *.*.*    *   *..*.*.*    *
         |                                                                  |
   10000 ++-----------------------------------------------------------------+



                                   interrupts.RES

   28000 ++---------------------------------------------------------------O-+
   26000 ++                                                                 |
         |                                                             O    |
   24000 ++     O                                                           |
   22000 ++                                                                 |
         |                                  O                               |
   20000 ++O                                                                |
   18000 ++                   O      O O        O                           |
   16000 ++                                                   O             |
         O        O                      O                                  |
   14000 ++   O      O          O             O    O               O        |
   12000 ++            O O O      O                  O    O O   O           |
         |     .*.*..                 .*.  .*.         O   .*.*.     O     .*
   10000 *+*..*      *.*.*.*..*.*.*..*   *.   *.*..*.*.*..*     *..*.*.  .* |
    8000 ++------------------------------------------------------------*----+



                                    interrupts.RES

   100000 ++----------------------------------------------------------------+
          |                                                                 |
    90000 ++                                                  O             |
    80000 ++                            O                                   |
          |          O    O                                                 |
    70000 ++                                                     O          |
          |                                                                 |
    60000 O+     O                                          O      O        |
          |                      O O O    O   O    O O  O                   *
    50000 ++O  O                 *          O             O                 |
    40000 *+       O    O   *   ::                                   O O   :|
          |:               : :  : :              *                         :|
    30000 ++  .*.          : :O:  :       *.   .. :       *.*.  .*.       * |
          | *.   *.      .*   *    :     +  *.*   :      +    *.   *.   ..  |
    20000 ++-------*-*--*----------*-*--*----------*-*--*------------*-*----+



                                   interrupts.RES

   26000 ++------------------O----------------------------------------------+
         |                                                                  |
   24000 ++                            O                                    |
   22000 ++O                                                 O       O      |
         |                               O       O    O                O    |
   20000 O+                         O                                       |
   18000 ++        O      O                                                 |
         |                                    O                             |
   16000 ++   O      O  O                          O       O                |
   14000 ++                       O         O                               |
         |      O  *                                              O       O |
   12000 ++      .. +          O                   *..  O    *..            O
   10000 ++    .*    *..*.       .*.    .*..*.    +   *.    +   O.*..       |
         *.*..*           *..*.*.   *..*      *..*      *..*    *    *.*..* |
    8000 ++-----------------------------------------------------------------+



                                   interrupts.RES

   29000 ++-----------------------------------------------------------------+
   28000 ++O                             O                                  |
         |                                  O   O           O               |
   27000 ++   O          O O  O   O           O      O O           O        |
   26000 O+          O          O    O             O      O   O        O    O
         |      O                      O                        O    O    O |
   25000 ++       O                                                         |
   24000 ++            O                                                    |
   23000 ++                                                                 |
         |                                                                  |
   22000 ++*..          .*.*..                            *.    *..*.       |
   21000 *+   *.*.*.. .*      *.*     .*.*..*.*.  .*.   ..  *. +     *.     |
         |           *           +  .*          *.   *.*      *        *..*.|
   20000 ++                       *.                                        *
   19000 ++-----------------------------------------------------------------+



                                    interrupts.RES

   4.5e+06 ++---------------------------------------------------------------+
           |                                                                |
     4e+06 ++O             O                   O                            |
           |        O            O      O             O            O        |
   3.5e+06 ++                                               O           O   |
           |                                                                |
     3e+06 ++                                                               |
           |                                                                |
   2.5e+06 ++     O                   O                          O          |
           O   O .*     *    O O *  O     O  O   * O  *.O O      *   O    O O
     2e+06 ++  *.  :  O: :       ::              ::  +  *      O ::     *   |
           *. +    :   :  :     :  :            : : +    +      : :    + :  |
   1.5e+06 ++*      *.*   : .*. :  : .*.        :  *      *.*.. :  :  +  : .*
           |               *   *    *   *.*..*.:               *   *.*    * |
     1e+06 ++----------------------------------*----------------------------+



                                    interrupts.RES

   1.4e+06 ++---------------------------------------------------------------+
           |                                 O                          O   |
   1.2e+06 ++     O              O             O            O               |
           |        O          O                          O    O          O |
           | O        O                          O    O            O        O
     1e+06 O+              O O      O                            O          |
           |            O             O O O        O    O            O      |
    800000 ++  O                                                            |
           |                                                                |
    600000 ++                                                               |
           |                                 *            *.            *.  |
           *.    .*. .*       .*           .. + .*.      +  *..        +  *.|
    400000 ++*.*.   *  +  .*.*  +  .*.*.*.*    *   *..*.*      *.*.*. +     *
           |            *.       *.                                  *      |
    200000 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                12772.48       +22.6%     15653.17  kbuildx/micro/netperf/120s-200%-TCP_CRR
                 9805.83       +60.2%     15705.88  kbuildx/micro/netperf/120s-200%-TCP_RR
                28540.31       +92.5%     54927.70  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                10007.52       +73.3%     17347.70  kbuildx/micro/netperf/120s-200%-UDP_RR
                21038.78       +25.4%     26390.06  lkp-a04/micro/netperf/120s-200%-TCP_RR
              1625071.24       +68.7%   2742159.71  vpx/micro/hackbench/1600%-process-pipe
               393941.93      +166.7%   1050766.61  vpx/micro/hackbench/1600%-threads-socket
              2101178.08       +86.7%   3922950.83  TOTAL interrupts.RES


                            lock_stat.&wq->wait.con-bounces

   800000 ++----------------------------------------------------------------+
          |                      O                                          |
   700000 ++       O                        O O             O             O |
          |                                               O            O    |
   600000 O+O  O O   O  O O O O    O O    O      O O O  O     O  O O O      |
   500000 ++                            O                                   O
          |                                                                 |
   400000 ++                                                                |
          |                                                                 |
   300000 ++                                                                |
   200000 ++                                                                |
          |         .*..                   .*.           .*.*.              |
   100000 *+*..*.*.*    *.*.*.*..*.*.*..*.*   *..*.*.*..*     *..*.*.*.*..*.*
          |                                                                 |
        0 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               114438.13      +421.2%    596429.73  vpx/micro/hackbench/1600%-threads-socket
               114438.13      +421.2%    596429.73  TOTAL lock_stat.&wq->wait.con-bounces


                            lock_stat.&wq->wait.contentions

   800000 ++----------------------------------------------------------------+
          |                      O                                          |
   700000 ++       O                        O O             O             O |
          |                                               O            O    |
   600000 O+O  O O   O  O O O O    O O    O      O O O  O     O  O O O      |
   500000 ++                            O                                   O
          |                                                                 |
   400000 ++                                                                |
          |                                                                 |
   300000 ++                                                                |
   200000 ++                                                                |
          |         .*..                   .*.           .*.*.              |
   100000 *+*..*.*.*    *.*.*.*..*.*.*..*.*   *..*.*.*..*     *..*.*.*.*..*.*
          |                                                                 |
        0 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               114474.03      +421.1%    596570.07  vpx/micro/hackbench/1600%-threads-socket
               114474.03      +421.1%    596570.07  TOTAL lock_stat.&wq->wait.contentions


             lock_stat.&(&u->lock)->rlock.contentions.unix_stream_sendmsg

   2.4e+06 ++---------------------------------------------------------------+
   2.2e+06 O+                O                        O          O          |
           | O O      O O  O     O  O     O  O   O O    O      O   O        |
     2e+06 ++     O O                 O O      O          O O        O  O O O
   1.8e+06 ++                  O                                            |
   1.6e+06 ++                                                               |
   1.4e+06 ++                                                               |
           |                                                                |
   1.2e+06 ++                                                               |
     1e+06 ++                                                               |
    800000 ++                                                               |
    600000 ++                                                               |
           *.        .*.                    .*.        .*.*.                |
    400000 ++*.*..*.*   *..*.*.*.*..*.*.*.*.   *.*.*..*     *..*.*.*.*..*.*.*
    200000 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               410398.80      +399.1%   2048451.37  vpx/micro/hackbench/1600%-threads-socket
               410398.80      +399.1%   2048451.37  TOTAL lock_stat.&(&u->lock)->rlock.contentions.unix_stream_sendmsg


                      lock_stat.&(&u->lock)->rlock.waittime-total

   1.4e+06 ++----------------------------------O----------------------------+
           |                                                                |
   1.2e+06 ++                                                               |
           |                                                                |
     1e+06 ++                                                               |
           |                                                                |
    800000 ++O                                                              |
           O   O      O O  O O      O O O O      O O  O O      O O O        O
    600000 ++                    O           O            O          O  O   |
           |      O O          O                            O             O |
    400000 ++                                                               |
           |                                                                |
    200000 ++        .*.                                                    |
           *.*.*..*.*   *..*.*.*.*..*.*.*.*..*.*.*.*..*.*.*.*..*.*.*.*..*.*.*
         0 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               131814.51      +412.1%    675043.67  vpx/micro/hackbench/1600%-threads-socket
               131814.51      +412.1%    675043.67  TOTAL lock_stat.&(&u->lock)->rlock.waittime-total


                       lock_stat.tasklist_lock-W.waittime-max

   40000 ++-----O-----------------------------------------------------------+
         |                                                                  |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   25000 ++                                                                 |
         |              O                                            O      |
   20000 O+                                                                 O
         |                                                                  |
   15000 ++                                                                 |
   10000 ++                                   O                             |
         |     .*...                                                        |
    5000 ++....     ...                               O   ...*.......    ...*
         *.            .    ...*.......O          ....*...           *...   |
       0 ++-------------*--------------*------*-----------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 3078.47      +395.2%     15244.22  lkp-snb01/micro/hackbench/1600%-process-socket
                 3078.47      +395.2%     15244.22  TOTAL lock_stat.tasklist_lock-W.waittime-max


                  lock_stat.&wq->wait.contentions.__wake_up_sync_key

   1.4e+06 ++--------------------O------------------------------------------+
           |        O                        O                              |
   1.2e+06 ++                                  O            O           O O |
           O   O  O   O O  O O O          O               O    O O          |
     1e+06 ++O                      O O O        O O  O O          O O      O
           |                                                                |
    800000 ++                                                               |
           |                                                                |
    600000 ++                                                               |
           |                                                                |
    400000 ++                                                               |
           |         .*.                    .*.          .*.*..             |
    200000 *+*.*..*.*   *..*.*.*.*..*.*.*.*.   *.*.*..*.*      *.*.*.*..*.*.*
           |                                                                |
         0 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               199966.47      +447.5%   1094797.30  vpx/micro/hackbench/1600%-threads-socket
               199966.47      +447.5%   1094797.30  TOTAL lock_stat.&wq->wait.contentions.__wake_up_sync_key


                    lock_stat.&(&list->lock)->rlock#2.acq-bounces

   60000 ++-----------------------------------------------------------------+
         |                                                                  |
   50000 ++                  O                                              |
         |                                                                  |
         |                                                           O      |
   40000 ++                                                  O              |
         |                          O  O         O                     O    |
   30000 ++             O                O                                  |
         O O       O O                                                      |
   20000 ++   O           O                   O                             |
         |                                            O                     |
         |                                         O                        |
   10000 ++        *              O                        O              O |
         |      O.. +                                .*.          O         |
       0 *+*--*-*----*--*-*--*-*--*-*--*-*--*-*--*-*----*--*-*--*-*--*-*--*-O

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  550.30     +3578.4%     20241.89  kbuildx/micro/netperf/120s-200%-UDP_RR
                  550.30     +3578.4%     20241.89  TOTAL lock_stat.&(&list->lock)->rlock#2.acq-bounces


                       lock_stat.&(&u->lock)->rlock.contentions

   1.4e+06 ++---------------------------------------------------------------+
           |                                                                |
   1.2e+06 O+                O               O        O          O          |
           | O O      O O  O     O  O     O      O O    O O    O   O    O   |
           |      O O                 O O      O            O        O    O O
     1e+06 ++                  O                                            |
           |                                                                |
    800000 ++                                                               |
           |                                                                |
    600000 ++                                                               |
           |                                                                |
           |                                                                |
    400000 ++         *                                                     |
           *.*.  .*. + +      .*.        .*..*. .*.  .*.*.*.*..        .*.*.|
    200000 ++--*----*---*--*-*---*--*-*-*------*---*-----------*-*-*-*------*

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               246919.83      +359.1%   1133680.47  vpx/micro/hackbench/1600%-threads-socket
               246919.83      +359.1%   1133680.47  TOTAL lock_stat.&(&u->lock)->rlock.contentions


                       lock_stat.&(&u->lock)->rlock.con-bounces

   1.4e+06 ++---------------------------------------------------------------+
           |                                                                |
   1.2e+06 O+                O               O        O          O          |
           | O O      O O  O     O  O     O      O O    O O    O   O    O   |
           |      O O                 O O      O            O        O    O O
     1e+06 ++                  O                                            |
           |                                                                |
    800000 ++                                                               |
           |                                                                |
    600000 ++                                                               |
           |                                                                |
           |                                                                |
    400000 ++         *                                                     |
           *.*.  .*. + +      .*.        .*..*. .*.  .*.*.*.*..        .*.*.|
    200000 ++--*----*---*--*-*---*--*-*-*------*---*-----------*-*-*-*------*

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               246778.20      +359.1%   1132916.40  vpx/micro/hackbench/1600%-threads-socket
               246778.20      +359.1%   1132916.40  TOTAL lock_stat.&(&u->lock)->rlock.con-bounces


               lock_stat.&af_unix_sk_receive_queue_lock_key.acq-bounces

   1.8e+08 ++---------------------------------------------------------------+
           |                     O                                          |
   1.6e+08 ++       O                          O                            |
   1.4e+08 ++                                O              O           O O |
           |   O  O          O                            O      O          |
   1.2e+08 O+O        O O  O   O    O O   O      O O  O O      O   O O      |
           |                            O                                   O
     1e+08 ++                                                               |
           |                                                                |
     8e+07 ++                                                               |
     6e+07 ++                                                               |
           |                                                                |
     4e+07 ++         *                                                     |
           |.*.  .*. + +  .*.*.*.  .*. .*.*..*. .*.  .*.*.*.*..   .*.*..*.*.|
     2e+07 *+--*----*---*--------*----*--------*---*-----------*-*----------*

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             27255657.90      +378.6% 130443670.13  vpx/micro/hackbench/1600%-threads-socket
             27255657.90      +378.6% 130443670.13  TOTAL lock_stat.&af_unix_sk_receive_queue_lock_key.acq-bounces


                       lock_stat.&(&u->lock)->rlock.acq-bounces

   1.8e+08 ++---------------------------------------------------------------+
           |                     O                                          |
   1.6e+08 ++       O                          O                            |
   1.4e+08 ++                                O              O           O O |
           |   O  O          O                            O      O          |
   1.2e+08 O+O        O O  O   O    O O   O      O O  O O      O   O O      |
           |                            O                                   O
     1e+08 ++                                                               |
           |                                                                |
     8e+07 ++                                                               |
     6e+07 ++                                                               |
           |                                                                |
     4e+07 ++         *                                                     |
           |.*.  .*. + +  .*.*.*.  .*. .*.*..*. .*.  .*.*.*.*..   .*.*..*.*.|
     2e+07 *+--*----*---*--------*----*--------*---*-----------*-*----------*

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             27262725.17      +378.5% 130460259.53  vpx/micro/hackbench/1600%-threads-socket
             27262725.17      +378.5% 130460259.53  TOTAL lock_stat.&(&u->lock)->rlock.acq-bounces


                          lock_stat.&wq->wait.waittime-total

     4e+06 ++---------------------------------------------------------------+
           O   O      O O  O O                        O        O   O        |
   3.5e+06 ++O                      O O O O      O O    O        O   O      O
           |                     O           O                              |
     3e+06 ++       O                          O            O             O |
           |      O            O                          O             O   |
   2.5e+06 ++                                                               |
           |                                                                |
     2e+06 ++                                                               |
           |                                                                |
   1.5e+06 ++                                                               |
           |                                                                |
     1e+06 ++        .*.                                        .*.         |
           *.*.*..*.*   *..*.*.*.*..*.*.*.*..*.*.*.*..*.*.*.*..*   *.*..*.*.*
    500000 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               783034.38      +332.0%   3382654.57  vpx/micro/hackbench/1600%-threads-socket
               783034.38      +332.0%   3382654.57  TOTAL lock_stat.&wq->wait.waittime-total


                   lock_stat.&wq->wait.contentions.prepare_to_wait

   80000 ++---------------------O-------------------------------------------+
         |        O                         O             O               O |
   70000 ++                                   O             O          O    |
         |      O             O                                             |
   60000 O+   O        O O O         O   O      O  O            O    O      |
         | O         O            O    O             O O      O    O        |
   50000 ++                                                                 O
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |                                                                  |
   20000 ++    .*.  .*.      .*.           .*.           .*.*.        .*..  |
         *.  .*   *.   *.*.*.   *.*..*.*.*.   *.*..*.*.*.     *.*..*.*    *.*
   10000 ++*----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                15499.47      +302.6%     62406.17  vpx/micro/hackbench/1600%-threads-socket
                15499.47      +302.6%     62406.17  TOTAL lock_stat.&wq->wait.contentions.prepare_to_wait


             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   3500 ++------------------------------------------------------------------+
        |                                                              O    |
   3000 ++                                                                  |
        | O    O                                                          O |
   2500 ++                                                                  |
        |                    O             O                                |
   2000 ++                            O                                     |
        O                                                                   |
   1500 ++                          O             O                         |
        |        O                              O    O                      |
   1000 ++            O                  O                    O             |
        |           O                         O                             |
    500 ++   O          O      O  O                      O                  |
        |                  O                           O    O   O  O O      |
      0 *+*--*-*-*--*-*-*--*-*-*--*-*-*--*-*--*-*-*--*-*-*--*-*-*--*-*-*--*-*



             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   14000 ++-----------------------------------------------------------------+
         |                                                                  |
   12000 ++                                                   O             |
         |                                                                  |
   10000 ++     O                                                           |
         |           O   O             O                        O           |
    8000 ++                       O           O                             |
         O                      O                         O                 *
    6000 ++                *    *           O      O O O    O      O       :|
         | O      O        O:   :    O   O      O         *                :|
    4000 ++   O        O  : :  : :              *        : :             .* |
         *.               :  :O: :             : :       : :         O *. O |
    2000 ++*..*.         :   ::  :             :  :     :   :         +     |
         |      *.    .*.*    *   : .*.*.*..*.*   : .*.*    *.*.*..*.*      |
       0 ++-------*--*------------*----------------*------------------------+



             lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base

   80000 ++-----------------------------------------------------------------+
         |    O                                                             |
   70000 ++O                    O                                  O        |
         |                               O      O                           |
         |           O     O      O  O             O      O O          O    |
   60000 ++       O      O    O             O        O        O      O    O |
         O      O      O                      O                             O
   50000 ++                            O               O           *        |
         |               *      *                               O + :       |
   40000 *+*            + +    ::      *    *      *             +  :       |
         |  +    .*.. .*   *.. : :    : + .. :    + +           *    :      |
         |   + .*    *        *  :    :  *   :   +   *.    .*. +     *.*..  |
   30000 ++   *                   : .*        *.*      *..*   *           *.|
         |                        *.                                        *
   20000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                   79.93     +1402.2%      1200.76  kbuildx/micro/netperf/120s-200%-TCP_RR
                 1905.73      +229.8%      6285.72  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                34416.57       +75.5%     60415.90  lkp-a04/micro/netperf/120s-200%-TCP_RR
                36402.23       +86.5%     67902.38  TOTAL lock_stat.&(&base->lock)->rlock.contentions.lock_timer_base


                           lock_stat.&wq->wait.acq-bounces

   40000 ++-----------------------------------------------------------------+
         |                                                             O    |
   35000 ++O                                                              O |
   30000 ++                   O                                             |
         |      O                      O    O                               |
   25000 ++                                                                 |
         |                                                                  |
   20000 O+                                                                 |
         |                           O   O      O             O             |
   15000 ++       O    O                           O                        |
   10000 ++                                                                 |
         |           O                        O      O    O                 |
    5000 ++   O          O      O O                                         |
         |                 O                           O    O   O  O O      |
       0 *+*--*-*-*--*-*-*-*--*-*-*--*-*-*--*-*-*--*-*-*--*-*-*-*--*-*-*--*-*



                            lock_stat.&wq->wait.acq-bounces

   350000 ++----------------------------------------------------------------+
          |                                                                 |
   300000 ++                                                  O             |
          |          O                                                      |
          |               O             O                        O          |
   250000 ++                                                                |
          |      O               O            O                             |
   200000 ++                       O      O             O O O               |
          O                                 O      O O             O        *
   150000 ++                *    *   O                                     :|
          | O  O        O   :   ::                                         :|
          *        O       : :O : :              *        *          O O .* |
   100000 ++               : : :  :       *.   .. :      : +          .*.   |
          | *..*.*. .*.. .*   ::   :     +  *.*   :      :  *.*..*.*.*      |
    50000 ++-------*----*-----*----*-*--*----------*-*--*-------------------+



                           lock_stat.&wq->wait.acq-bounces

   60000 ++-----------------------------------------------------------------+
         |                                                                  |
   50000 ++                  O                                              |
         |                                                                  |
         |                                                           O      |
   40000 ++                                                  O              |
         |                          O  O         O                     O    |
   30000 ++             O                O                                  |
         O O       O O                                                      |
   20000 ++   O           O                   O                             |
         |                                            O                     |
         |                                         O                        |
   10000 ++        *              O                        O              O |
         |      O.. +                                .*.          O         |
       0 *+*--*-*----*--*-*--*-*--*-*--*-*--*-*--*-*----*--*-*--*-*--*-*--*-O



                            lock_stat.&wq->wait.acq-bounces

   150000 ++----------------------------------------------------------------+
   140000 ++   O                                                            |
          | O                                    O                          |
   130000 ++                     O O      O                 O      O        |
   120000 ++         O    O O O      O      O      O O    O            O    |
   110000 ++       O                          O         O     O      O      |
   100000 O+     O      O               O                                 O O
          |                                                      O          |
    90000 ++                                                       *        |
    80000 ++                                                      + :       |
    70000 *+*..          .*.     *      *. .*      *.            *  :       |
    60000 ++    .*.*.  .*   *. .. :   ..  *  +    +  *..   .*. ..    *.     |
          |    *     *.       *   : .*        *..*      *.*   *        *..*.|
    50000 ++                       *                                        *
    40000 ++----------------------------------------------------------------+



                           lock_stat.&wq->wait.acq-bounces

   8e+06 ++-----------------------------------------------------------------+
         |                                  O                               |
   7e+06 ++                     O             O             O          O    |
         |      O O           O                           O   O             |
   6e+06 ++          O   O                      O    O             O      O O
         O O  O        O   O      O  O O               O        O    O      |
   5e+06 ++                              O         O                        |
         |                                                                  |
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                 *            .*.                |
         *.    .*.  .*.      .*.          .. + .*..   .*.   *.        .*..*.|
   2e+06 ++*..*   *.   *.*.*.   *.*..*.*.*    *    *.*        *.*..*.*      *
         |                                                                  |
   1e+06 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  196.63     +7498.6%     14941.38  kbuildx/micro/netperf/120s-200%-TCP_RR
                82296.63      +123.4%    183823.69  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                  570.63     +3443.1%     20217.93  kbuildx/micro/netperf/120s-200%-UDP_RR
                61253.43       +89.6%    116134.40  lkp-a04/micro/netperf/120s-200%-TCP_RR
              2231794.03      +169.5%   6013890.20  vpx/micro/hackbench/1600%-threads-socket
              2376111.36      +167.2%   6349007.60  TOTAL lock_stat.&wq->wait.acq-bounces


                          lock_stat.&pipe->wait.contentions

   50000 ++-----------------------------------------------------------------+
         |                             O      O                             |
   45000 ++                                                                 |
   40000 ++                                                                 |
         |                                                                  |
   35000 ++O                                                       O   O    |
   30000 ++                     O                                           |
         |        O      O                           O O        O           O
   25000 ++   O                   O  O                      O        O      |
   20000 ++   *        O   O                                                |
         O   + :O    O        O                 O  O      O                 |
   15000 ++ +  :                *        O  O   *..           O        *  O |
   10000 ++*    :              + +  .*         :                      : +   |
         |+     *.*..*.*.*.*..*   *.  + .*..*. :   *.    .*.*.*.  .*. :  + .*
    5000 *+----------------------------*------*------*-*--------*----*----*-+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 8354.77      +192.5%     24438.10  vpx/micro/hackbench/1600%-process-pipe
                 8354.77      +192.5%     24438.10  TOTAL lock_stat.&pipe->wait.contentions


                          lock_stat.&pipe->wait.con-bounces

   50000 ++-----------------------------------------------------------------+
         |                             O      O                             |
   45000 ++                                                                 |
   40000 ++                                                                 |
         |                                                                  |
   35000 ++O                                                       O   O    |
   30000 ++                     O                                           |
         |        O      O                           O O        O           O
   25000 ++   O                   O  O                      O        O      |
   20000 ++   *        O   O                                                |
         O   + :O    O        O                 O  O      O                 |
   15000 ++ +  :                *        O  O   *..           O        *  O |
   10000 ++*    :              + +  .*         :                      : +   |
         |+     *.*..*.*.*.*..*   *.  + .*..*. :   *.    .*.*.*.  .*. :  + .*
    5000 *+----------------------------*------*------*-*--------*----*----*-+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 8350.63      +192.4%     24415.77  vpx/micro/hackbench/1600%-process-pipe
                 8350.63      +192.4%     24415.77  TOTAL lock_stat.&pipe->wait.con-bounces


                 lock_stat.&pipe->wait.contentions.__wake_up_sync_key

   100000 ++----------------------------------------------------------------+
          |                                                                 |
    90000 ++                            O     O                             |
    80000 ++                                                                |
          |                                                                 |
    70000 ++O                                                               |
    60000 ++                                                       O   O    |
          |        O      O      O                   O           O          O
    50000 ++   O                   O                    O   O        O      |
    40000 ++   *        O   O        O                                      |
          O   + :             O                  O O      O                 |
    30000 ++ +  :O   O           *        O O    *            O           O |
    20000 ++*    :             .. :  *..        + +                    *..  |
          |+     *.*.*..*. .*.*   : +    .*.*. +   *.    .*.*.*.. .*. +    .*
    10000 *+--------------*--------*----*-----*------*--*--------*---*----*-+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                16417.70      +186.8%     47080.03  vpx/micro/hackbench/1600%-process-pipe
                16417.70      +186.8%     47080.03  TOTAL lock_stat.&pipe->wait.contentions.__wake_up_sync_key


                     lock_stat.&(&base->lock)->rlock.acq-bounces

   80000 ++-----------------------------------------------------------------+
         |                                                             O  O |
   70000 ++O                                                                |
   60000 ++                   O                                             |
         |      O                           O                               |
   50000 ++                            O                                    |
         |                                                                  |
   40000 O+                                                                 |
         |                               O      O                           |
   30000 ++            O             O             O          O             |
   20000 ++       O                                                         |
         |           O                        O      O    O                 |
   10000 ++   O          O      O O                                         |
         |                 O                           O    O   O  O O      |
       0 *+*--*-*-*--*-*-*-*--*-*-*--*-*-*--*-*-*--*-*-*--*-*-*-*--*-*-*--*-*



                      lock_stat.&(&base->lock)->rlock.acq-bounces

   900000 ++----------------------------------------------------------------+
          |                                                   O             |
   800000 ++                                                                |
   700000 ++         O                                                      |
          |               O             O                        O          |
   600000 ++     O                                                          |
   500000 ++                     O O          O         O O O               |
          O                               O O      O O             O        *
   400000 ++O               O    *   O                                     :|
   300000 ++   O   O    O   *   ::               O        *                :|
          *                 ::O : :              *        ::         O O .* |
   200000 ++               : : :  :             + :      : :          .*.   |
   100000 ++*..*.          :  ::   :       .*. +  :      :  *.  .*. .*      |
          |      *.*.*..*.*   *    *.*..*.*   *    *.*..*     *.   *        |
        0 ++----------------------------------------------------------------+



                      lock_stat.&(&base->lock)->rlock.acq-bounces

   280000 ++---O------------------------------------------------------------+
   260000 ++O                                                               |
          |                      O        O      O                 O        |
   240000 ++         O      O      O        O      O      O O               |
   220000 ++       O      O   O      O               O        O      O O    |
          |      O                            O         O                 O |
   200000 O+            O               O                        O          O
   180000 ++                                                                |
   160000 ++                                                       *        |
          |               *.     *      *   *                     + :       |
   140000 *+*..   .*.    +  *. .. :    + + + :     *.            *  :       |
   120000 ++    .*   *..*     *   :   +   *  :    +  *..   .*. ..    *.     |
          |    *                   :.*        *..*      *.*   *        *..*.|
   100000 ++                       *                                        *
    80000 ++----------------------------------------------------------------+



                      lock_stat.&(&base->lock)->rlock.acq-bounces

   340000 ++----------------------------------------------------------------+
          |                                                                 |
   330000 ++                              O                                 |
          |          O      O O               O         O O      O          |
   320000 ++                       O    O        O   O      O O      O    O O
          O O  O O O    O        O                 O                   O    |
   310000 ++              O          O      O                      O        |
          |                                                                 |
   300000 ++                                                                |
          |                                                                 |
   290000 ++                                                                |
          |   .*           .*.*..                                           |
   280000 ++*.  + .*.*.. .*      *. .*.. .*.*.*..*.*.  .*   *.*..*. .*.*..*.|
          *      *      *          *    *            *.  + +       *        *
   270000 ++----------------------------------------------*-----------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 1490.47     +1866.4%     29308.55  kbuildx/micro/netperf/120s-200%-TCP_RR
               142602.93      +218.6%    454270.66  kbuildx/micro/netperf/120s-200%-TCP_STREAM
               124451.17       +82.5%    227061.80  lkp-a04/micro/netperf/120s-200%-TCP_RR
               279009.90       +14.1%    318426.10  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               547554.47       +87.9%   1029067.11  TOTAL lock_stat.&(&base->lock)->rlock.acq-bounces


              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   4000 ++------------------------------------------------------------------+
        |                                                 O    O            |
   3500 ++                        O                                         O
   3000 ++              O                         O                 O       |
        |         O                                               O         |
   2500 ++   O                  O    O                      O               |
        *                 O                         O                       |
   2000 ++ O    *                            O O                            |
        |:    .. +           O                                         O    |
   1500 ++   *    *               *       O                                 |
   1000 ++:  :     +    *.*      : :   O          *.   O                    |
        | : :       +O..   :     :  :            +  *..*..     *            |
    500 ++ ::   O    *      :   *   :           +         *. .. +     .*.O  |
        O  *                : ..     *.*..*..*.*            *    + .*.   *  |
      0 ++-------------------*------------------------------------*---------+



              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   4500 ++-----------------O------------------O-----------------------------+
        |                                                              O    |
   4000 ++O                  O O         O O                                |
   3500 ++            O                                  O           O    O |
        O                           O                  O           O        |
   3000 ++                                                                  |
   2500 ++                                                  O               |
        |                                         O  O                      |
   2000 ++   O      O                                                       |
   1500 ++              O                            *.                     |
        |    *                    *.*.*    *..     ..  *        O     .*    |
   1000 *+  + +O                 +     +  :   * O *     :    .*.  .*.*  +   |
    500 ++ +   *.O .*.*.*..*.*. + O     + :    + +      :  .*   *.       + .*
        | *      *.            *         *      *        *.               * |
      0 ++------------------------------------------------------------------+



              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   85000 ++-----------------------------------------------------------------+
         |      O                    O   O                O                 O
   80000 O+O         O   O    O             O                   O         O |
   75000 ++       O    O   O                           O           O O      |
         |                      O O           O      O        O             |
   70000 ++                            O        O  O        O          O    |
         |    O                                                             |
   65000 ++                                                                 |
         |                                                                  |
   60000 ++                                                                 |
   55000 ++              *.                                 *.*    *      * |
         *.  .*         +  *..       *                 *.. +   + .. :   ..  |
   50000 ++*.  +       *            + + .*..          +   *     *   : .*    |
         |      *.*.. +       *.*. +   *     .*.  .*.*               *      |
   45000 ++----------*------------*---------*---*---------------------------+



              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   35000 ++-----------------------------------------------------------------+
         |                                                                  |
   30000 ++                       O                             O           |
         | O  O      O O   O    O      O O  O O    O O                      |
   25000 O+     O O      O    O      O          O      O  O O O    O O O  O O
         |                                                                  |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++                                                                 |
         |                                                                  |
   10000 *+      .*..*       .*.                  .*. .*.. .*.*    *.*    *.|
         | *..*.*     + .*.*.   *.*..*.*.*.. .*.*.   *    *    + ..   + ..  *
    5000 ++            *                    *                   *      *    |
         |                                                                  |
       0 ++-----------------------------------------------------------------+



              lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total

   90000 ++-----------------------------------------------------------------+
   85000 ++                                        O        O               |
         |                           O                                      O
   80000 ++                                                     O           |
   75000 ++O           O O                           O O      O        O    |
         |      O O  O     O    O O    O    O O O                  O      O |
   70000 ++   O               O          O                O          O      |
   65000 O+                                                                 |
   60000 ++                                                                 |
         |                                                                  |
   55000 ++                                                                 |
   50000 *+                                        *.*.    .*.    .*.*.    .*
         | *..*.*.  .*. .*.*..*.*.*..*.*.*..      +    *..*   *.*.     *..* |
   45000 ++       *.   *                    *.*. +                          |
   40000 ++-------------------------------------*---------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  767.25      +194.9%      2262.75  kbuildx/micro/netperf/120s-200%-TCP_MAERTS
                  811.51      +232.3%      2696.42  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                50298.50       +52.0%     76470.48  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 8197.96      +221.7%     26369.07  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                47199.11       +57.0%     74099.12  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               107274.33       +69.6%    181897.84  TOTAL lock_stat.((&icsk->icsk_retransmit_timer)).holdtime-total


                     lock_stat.&wq->wait.contentions.finish_wait

   50000 ++-----------------------------------------------------------------+
         |                      O                                           |
   45000 ++       O                                         O               |
   40000 ++                                   O           O               O |
         |      O                           O                          O    |
   35000 ++   O               O      O   O                      O           |
         O O         O O O O                    O  O   O             O      |
   30000 ++                       O    O             O        O    O        O
         |                                                                  |
   25000 ++                                                                 |
   20000 ++                                                                 |
         |                                                                  |
   15000 ++     *.  .*.      .*.            *. .*..       *.*.        .*..*.|
         *.    +  *.   *.*.*.   *.*..*.   ..  *    *.*. ..    *.*.. .*      *
   10000 ++*--*------------------------*-*-------------*-----------*--------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                12952.33      +173.5%     35419.90  vpx/micro/hackbench/1600%-threads-socket
                12952.33      +173.5%     35419.90  TOTAL lock_stat.&wq->wait.contentions.finish_wait


                    lock_stat.&pipe->mutex/1.contentions.pipe_read

   140000 ++----------------------------------------------------------------+
          |                                                                 |
   120000 ++       O                    O     O      O             O        |
          | O             O      O                          O               |
   100000 ++                                                           O    |
          |                                                                 |
    80000 ++                                                                |
          |                                                                 |
    60000 O+   O O   O  O   O O    O O    O O    O O    O O   O  O   O      O
          |                                                               O |
    40000 ++                                                                |
          *.  .*.*.    .*.      .*.             .*. .*..*.      .*.   .*..  |
    20000 ++*.     *.*.   *.*.*.   *.*..*.*.*.*.   *      *.*.*.   *.*    *.*
          |                                                                 |
        0 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                23590.80      +226.2%     76954.50  vpx/micro/hackbench/1600%-process-pipe
                23590.80      +226.2%     76954.50  TOTAL lock_stat.&pipe->mutex/1.contentions.pipe_read


                         lock_stat.&pipe->mutex/1.contentions

   160000 ++----------------------------------O-----------------------------+
          |                                                                 |
   140000 ++       O                    O            O             O        |
          | O             O      O                          O          O    |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++   O O      O     O    O O           O        O      O   O      O
          O          O      O             O O      O    O     O           O |
    60000 ++                                                                |
          |                                                                 |
    40000 ++  .*.      .*.      .*.             .*.                   .*..  |
          *.*.   *.*.*.   *.*.*.   *.*..*.*.*.*.   *.*..*.*.*.*..*.*.*    *.*
    20000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                29976.37      +219.6%     95797.37  vpx/micro/hackbench/1600%-process-pipe
                29976.37      +219.6%     95797.37  TOTAL lock_stat.&pipe->mutex/1.contentions


                         lock_stat.&pipe->mutex/1.con-bounces

   160000 ++----------------------------------O-----------------------------+
          |                                                                 |
   140000 ++       O                    O            O             O        |
          | O             O      O                          O          O    |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
    80000 ++   O O      O     O    O O           O        O      O   O      O
          O          O      O             O O      O    O     O           O |
    60000 ++                                                                |
          |                                                                 |
    40000 ++  .*.      .*.      .*.             .*.                   .*..  |
          *.*.   *.*.*.   *.*.*.   *.*..*.*.*.*.   *.*..*.*.*.*..*.*.*    *.*
    20000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                29976.37      +219.6%     95797.37  vpx/micro/hackbench/1600%-process-pipe
                29976.37      +219.6%     95797.37  TOTAL lock_stat.&pipe->mutex/1.con-bounces


                    lock_stat.&pipe->mutex/1.contentions.pipe_write

   180000 ++----------------------------------------------------------------+
          |                                   O                             |
   160000 ++                            O                                   |
   140000 ++O      O             O                   O             O        |
          |               O                                 O          O    |
   120000 ++                                                                |
          |                                                                 |
   100000 ++                                                                O
          |    O O      O     O    O O           O        O      O   O      |
    80000 O+         O      O             O O      O    O     O           O |
    60000 ++                                                                |
          |    *                                                            |
    40000 ++ .. +      .*.      .*.             .*.                   .*..  |
          *.*    *.*.*.   *.*.*.   *.*..*.*.*.*.   *.*..*.*.*.*..*.*.*    *.*
    20000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                33713.07      +208.6%    104030.90  vpx/micro/hackbench/1600%-process-pipe
                33713.07      +208.6%    104030.90  TOTAL lock_stat.&pipe->mutex/1.contentions.pipe_write


                   lock_stat.&(&base->lock)->rlock.waittime-total

   6000 ++------------------------------------------------------------------+
        |                                O                                  |
   5000 ++                 O                                                |
        |                                                                   |
        |                                                     *             |
   4000 ++          O                                *        :             |
        |      O        O                            :        :             |
   3000 ++                            O             : :O     : :            |
        |                         O O      O  O     : :      : :O           |
   2000 ++                     *                    :O:      : :            *
        O        O         *   ::                 O:  :  *  O: :   O       +|
        * O  O        O   + : : :               *  :   :: + :   :        .* |
   1000 ++               +  :O:  :             + : :   ::  +:   :    O.*. O |
        | *..*.        .*    *   :   .*.. .*..*  ::    *    *   *..*.*      |
      0 ++-----*-*--*-*-----------*-*----*--------*-------------------------+



                    lock_stat.&(&base->lock)->rlock.waittime-total

   140000 ++----------------------------------------------------------------+
   130000 ++   O                                                            |
          |                                                        O        |
   120000 ++O                    O        O      O                          |
   110000 ++         O      O      O O             O                        |
          |        O          O             O        O    O O          O    |
   100000 ++              O                   O               O      O    O |
    90000 O+     O      O               O               O          *        O
    80000 ++                                                     O::        |
          |               *      *      *                         : :       |
    70000 *+*            + +    + :    : + .*      *.            *  :       |
    60000 ++ +   *.*.  .*   *. +  :    :  *  :    +  *..   .*. ..    :      |
          |   + +    *.       *    :  :      :  .*       .*   *      *.*..  |
    50000 ++   *                   *.*        *.        *                 *.|
    40000 ++----------------------------------------------------------------*



                   lock_stat.&(&base->lock)->rlock.waittime-total

   8500 ++------------------------------------------------------O-----------+
        |           O                                    O                  |
   8000 ++O                  O        O  O           O        O      O      |
        O    O   O    O                    O  O        O               O  O |
        |      O        O      O  O O             O                O        |
   7500 ++                 O                    O                           O
        |                                                   O               |
   7000 ++                                                                  |
        |                  *       .*.    .*..    *..*      *.       *.     |
   6500 *+*..   .*..*.   .. + .*..*   *..*       +    +    :  *.    +  *..  |
        |    *.*      *.*    *                *.*      *   :    *..*      *.*
        |                                               + :                 |
   6000 ++                                               *                  |
        |                                                                   |
   5500 ++------------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  873.59      +169.4%      2353.36  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                59964.29       +74.1%    104409.31  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 6518.87       +19.9%      7814.32  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                67356.75       +70.1%    114576.99  TOTAL lock_stat.&(&base->lock)->rlock.waittime-total


                        lock_stat.&rq->lock/1.waittime-total

   18000 *+-----------------------------------------------------------------+
         |                                                                  |
   16000 ++                                                                 |
   14000 ++                                                                 |
         |:                                                                 |
   12000 ++                   O                                             |
   10000 ++                                                               O |
         |:            O                                                    |
    8000 ++O                                                                |
    6000 ++:  O   *        O    O               O         O   O             |
         | :      O+              O  O   O    O      O O             O      O
    4000 ++*..  O:  +    O             O               *..*     O      *..  |
    2000 O+   *. :   *.      .*.*.  .*.     O         +    +O      O  +   *.*
         |      *    O *.*.*.     *.   *.*..*. .*..O.*      *.*.*..*.*      |
       0 ++-----------------------------------*----*------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 2546.17      +104.8%      5213.65  vpx/micro/hackbench/1600%-process-pipe
                 2546.17      +104.8%      5213.65  TOTAL lock_stat.&rq->lock/1.waittime-total


             lock_stat.&(&u->lock)->rlock.contentions.unix_stream_recvmsg

   280000 ++----------------------------------------------------------------+
   260000 ++                     O            O                             |
          |        O                                        O               |
   240000 ++                                O                          O  O |
   220000 O+O    O        O O O    O O               O  O O   O  O O        |
   200000 ++   O     O  O               O O      O O                 O      O
   180000 ++                                                                |
          |                                                                 |
   160000 ++                                                                |
   140000 ++                                                                |
   120000 ++                                                                |
   100000 ++         *..                                                    |
          |     .*. +        .*.. .*.    .*.*.  .*. .*..*.*.*.    .*. .*..*.|
    80000 *+*..*   *    *.*.*    *   *..*     *.   *          *..*   *      *
    60000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                83441.20      +162.4%    218911.60  vpx/micro/hackbench/1600%-threads-socket
                83441.20      +162.4%    218911.60  TOTAL lock_stat.&(&u->lock)->rlock.contentions.unix_stream_recvmsg


             lock_stat.&af_unix_sk_receive_queue_lock_key.waittime-total

   50000 ++-----------------------------------------------------------------+
         |                                    O                             |
   45000 ++                                                                 |
   40000 ++                                                                 |
         |                                                                  |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |                                                                  |
   25000 O+                                            O      O O  O        |
   20000 ++O  O      O O O O      O  O O        O    O               O      O
         |      O O             O        O  O      O        O          O  O |
   15000 ++                   O                           O                 |
   10000 ++               .*..       *.                                     |
         *.*..*.*.*..*.*.*    *.*. ..  *.*..*.*.*..*.*.*..*. .*.*..*.*.    .*
    5000 ++-----------------------*-------------------------*----------*--*-+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 8189.34      +159.4%     21243.50  vpx/micro/hackbench/1600%-threads-socket
                 8189.34      +159.4%     21243.50  TOTAL lock_stat.&af_unix_sk_receive_queue_lock_key.waittime-total


                        lock_stat.&pipe->mutex/1.waittime-total

     3e+10 ++----------------------------------O----------------------------+
           |        O            O      O             O            O        |
           | O             O                                O           O   |
   2.5e+10 ++                                                               |
           O      O   O O      O    O O   O  O   O O    O O      O   O      O
           |   O             O                                 O          O |
     2e+10 ++                                                               |
           |                                                                |
   1.5e+10 ++                                                               |
           |                                                                |
           |                                                                |
     1e+10 ++        .*.             .*.*.        .*..              .*..    |
           *.*.*..*.*   *..*.*.*.*..*     *..*.*.*    *.*.*.*..*.*.*    *.*.*
           |                                                                |
     5e+09 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
           9002422305.74      +168.8% 24198678784.69  vpx/micro/hackbench/1600%-process-pipe
           9002422305.74      +168.8% 24198678784.69  TOTAL lock_stat.&pipe->mutex/1.waittime-total


                     lock_stat.&rq->lock.contentions.__schedule

   70000 ++-----------------------------------------------------------------+
         |                                    O                             |
   60000 ++       O             O                           O               |
         |    O O             O   O  O      O   O    O O        O  O O O    O
   50000 O+O         O O O O                       O      O   O             |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
   30000 ++                                                                 |
         |           *.*.*.         .*.                             .*.  .*.|
   20000 *+*..*     :      *..*   *.   *.*..*   *..*.*.*..*.*.*.*..*   *.   *
         |     :    :          :  :          :  :                           |
   10000 ++    :   :           : :           : :                            |
         |      :  :            ::     O      ::                            |
       0 ++-----*-*-------------*--------O----*---------------------------O-+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                18788.20      +162.0%     49224.97  vpx/micro/hackbench/1600%-process-pipe
                18788.20      +162.0%     49224.97  TOTAL lock_stat.&rq->lock.contentions.__schedule


         lock_stat.&af_unix_sk_receive_queue_lock_key.contentions.skb_unlink

   80000 ++-----------------------------------------------------------------+
         O                      O O         O O      O        O O  O        |
   70000 ++O  O   O  O O O O                    O      O    O          O    |
         |      O             O      O O O         O      O          O    O O
         |                                                                  |
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
         |          .*.              *. .*..        .*.*..        .*.       |
   30000 *+*.. .*.*.   *.*.*..*.*. ..  *    *.*.*..*      *. .*.*.   *.*.. .*
         |    *                   *                         *             * |
   20000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                28929.50      +146.7%     71357.03  vpx/micro/hackbench/1600%-threads-socket
                28929.50      +146.7%     71357.03  TOTAL lock_stat.&af_unix_sk_receive_queue_lock_key.contentions.skb_unlink


              lock_stat.&af_unix_sk_receive_queue_lock_key.contentions

   80000 ++-----------------------------------------------------------------+
         O                      O O         O O      O        O O  O        |
   70000 ++O  O   O  O O O O         O          O      O    O          O    |
         |      O             O        O O         O      O          O    O O
         |                                                                  |
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
         |          .*.              *. .*..        .*.*..        .*.       |
   30000 *+*.. .*.*.   *.*.*..*.*. ..  *    *.*.*..*      *. .*.*.   *.*.. .*
         |    *                   *                         *             * |
   20000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                28933.57      +146.6%     71363.73  vpx/micro/hackbench/1600%-threads-socket
                28933.57      +146.6%     71363.73  TOTAL lock_stat.&af_unix_sk_receive_queue_lock_key.contentions


              lock_stat.&af_unix_sk_receive_queue_lock_key.con-bounces

   80000 ++-----------------------------------------------------------------+
         O                      O O         O O      O        O O  O        |
   70000 ++O  O   O  O O O O         O          O      O    O          O    |
         |      O             O        O O         O      O          O    O O
         |                                                                  |
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
         |          .*.              *. .*..        .*.*..        .*.       |
   30000 *+*.. .*.*.   *.*.*..*.*. ..  *    *.*.*..*      *. .*.*.   *.*.. .*
         |    *                   *                         *             * |
   20000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                28932.37      +146.6%     71361.33  vpx/micro/hackbench/1600%-threads-socket
                28932.37      +146.6%     71361.33  TOTAL lock_stat.&af_unix_sk_receive_queue_lock_key.con-bounces


       lock_stat.&af_unix_sk_receive_queue_lock_key.contentions.skb_queue_tail

   80000 ++-----------------------------------------------------------------+
         O                      O O         O O      O        O O  O        |
   70000 ++O  O   O  O O O O         O          O      O    O          O    |
         |      O             O        O O         O      O          O    O O
         |                                                                  |
   60000 ++                                                                 |
         |                                                                  |
   50000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
         |                                                                  |
         |          .*.              *. .*..        .*.*..        .*.       |
   30000 *+*.. .*.*.   *.*.*..*.*. ..  *    *.*.*..*      *. .*.*.   *.*.. .*
         |    *                   *                         *             * |
   20000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                28937.70      +146.6%     71370.63  vpx/micro/hackbench/1600%-threads-socket
                28937.70      +146.6%     71370.63  TOTAL lock_stat.&af_unix_sk_receive_queue_lock_key.contentions.skb_queue_tail


                          lock_stat.slock-AF_INET.acq-bounces

   1.4e+06 ++--------------------------------------------------------O------+
           *                                          O O                   |
   1.2e+06 ++                                              O O              |
           |:                     O O                                       |
           |:             *..                 O                   O    O    |
     1e+06 ++   O  O O    :                 O    *.*            *           O
           | :  *..*.*   :   *         O        +   +           :           |
    800000 O+:  :     :  :   O+O               + O   +         : :          |
           | :O:       :O:     *    *         *    O  *.*..    : :          |
    600000 ++: :       ::       :   ::   O    :            *  :  :          |
           |  :         *        : :  :      :              + :  :        O |
           |  *                  : :  :      :               *    :         |
    400000 ++                     *    *.  .*                     : .*.*..* |
           |                             *.                       *.        |
    200000 ++---------------------------------------------------------------+



                         lock_stat.slock-AF_INET.acq-bounces

   60000 ++-----------------------------------------------------------------+
         |                                                                  |
   50000 ++                  O                                              |
         |                                                                  |
         |                                                           O      |
   40000 ++                                                  O              |
         |                          O  O         O                     O    |
   30000 ++             O                O                                  |
         O O       O O                                                      |
   20000 ++   O           O                   O                             |
         |                                            O                     |
         |                                         O                        |
   10000 ++        *              O                        O              O |
         |      O.. +                                .*.          O         |
       0 *+*--*-*----*--*-*--*-*--*-*--*-*--*-*--*-*----*--*-*--*-*--*-*--*-O



                         lock_stat.slock-AF_INET.acq-bounces

   75000 ++-----------------------------------------------------------------+
   70000 ++   O                                                             |
         | O                             O      O                           |
   65000 ++                     O O                         O      O        |
   60000 ++       O  O   O O  O      O      O      O O    O   O        O    |
   55000 ++                                   O        O             O      |
   50000 O+     O      O               O                                  O O
         |                                                      O           |
   45000 ++                                                        *        |
   40000 ++                                                      .. :       |
   35000 *+*..          .*.     *      *.  .*      *.           *   :       |
   30000 ++    .*.*.. .*   *.. + :    +  *.  +   ..  *.    .*. +     *.     |
         |    *      *        *  :  .*        *.*      *..*   *        *..*.|
   25000 ++                       *.                                        *
   20000 ++-----------------------------------------------------------------+



                          lock_stat.slock-AF_INET.acq-bounces

   270000 ++------------------------------O---------------------------------+
          |                                                                 |
   260000 ++         O      O O         O     O         O O      O          |
          |      O O    O          O             O O O      O O      O    O O
          O O  O          O      O                                     O    |
   250000 ++                         O      O                      O        |
          |                                                                 |
   240000 ++                                                                |
          |                                                                 |
   230000 ++                                                                |
          |   .*           .*.                               .*..           |
          |.*.  + .*.*.. .*   *..*. .*.. .*.*.  .*.*.  .*   *    *. .*.*..*.|
   220000 *+     *      *          *    *     *.     *.  + +       *        *
          |                                               *                 |
   210000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               658735.77       +47.4%    971176.00  kbuildx/micro/netperf/120s-200%-TCP_MAERTS
                  550.30     +3578.5%     20242.39  kbuildx/micro/netperf/120s-200%-UDP_RR
                31016.63       +89.5%     58767.27  lkp-a04/micro/netperf/120s-200%-TCP_RR
               221888.77       +15.9%    257094.67  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               912191.47       +43.3%   1307280.33  TOTAL lock_stat.slock-AF_INET.acq-bounces


                   lock_stat.&mapping->i_mmap_mutex.waittime-total

   8e+06 ++-----------------------------------------------------------------+
         |                                            O                     |
   7e+06 ++                                                                 |
   6e+06 ++                                                                 |
         |                                                                  |
   5e+06 ++                                                                 |
         |                                                                  |
   4e+06 ++                                                                 |
         |            ..*..                                                 |
   3e+06 ++     O ....     .                  O                      O      |
   2e+06 O+    .*.          .                                               |
         |   ..              ..O     ..*.....                O              |
   1e+06 ++..                    ....  O     .    ....*......*.......*..... O
         *.                    *.             *...                         .|
       0 ++-----------------------------------------------------------------*

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              1244708.30      +122.6%   2770689.24  lkp-snb01/micro/hackbench/1600%-threads-pipe
              1244708.30      +122.6%   2770689.24  TOTAL lock_stat.&mapping->i_mmap_mutex.waittime-total


                           lock_stat.&rq->lock.contentions

   45000 ++-----------------------------------------------------------------+
         |    O                                                             |
   40000 ++O                                                       O        |
         |                      O        O      O                           |
   35000 ++          O     O      O  O             O      O O          O    |
         |        O      O    O             O        O        O      O    O |
   30000 O+     O      O                      O        O           *        O
         |                             O                          + :       |
   25000 *+             .*      *      *                        O+  :       |
         | *..         *  +    : :    : +  .*      *.           *    :      |
   20000 ++    .*.*.. +    *.. : :    :  *.  +   ..  *.    .*. +     *.     |
         |    *      *        *   : .*        *.*      *..*   *        *..  |
   15000 ++                       *.                                      *.|
         |                                                                  *
   10000 ++-----------------------------------------------------------------+



                           lock_stat.&rq->lock.contentions

   75000 ++-----------------------------------------------------------------+
   70000 ++                                   O                             |
         |                      O                                           |
   65000 ++       O                                         O               |
   60000 ++     O             O      O O    O        O          O  O   O    O
   55000 O+O  O      O O O O      O             O  O   O  O          O      |
   50000 ++                              O                    O           O |
         |                                                                  |
   45000 ++                                                                 |
   40000 ++                                                                 |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |            .*.*.  .*.    .*.        .*..                         |
   25000 *+*..*.*.*..*     *.   *.*.   *.*..*.*    *. .*..*.*.*.*..*.*.*..*.*
   20000 ++------------------------------------------*----------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                19932.93       +69.5%     33789.80  lkp-a04/micro/netperf/120s-200%-TCP_RR
                24608.77      +133.8%     57545.60  vpx/micro/hackbench/1600%-process-pipe
                44541.70      +105.1%     91335.40  TOTAL lock_stat.&rq->lock.contentions


                           lock_stat.&rq->lock.con-bounces

   45000 ++-----------------------------------------------------------------+
         |    O                                                             |
   40000 ++O                                                       O        |
         |                      O        O      O                           |
   35000 ++          O     O      O  O             O      O O          O    |
         |        O      O    O             O        O        O      O    O |
   30000 O+     O      O                      O        O           *        O
         |                             O                          + :       |
   25000 *+             .*      *      *                        O+  :       |
         | *..         *  +    : :    : +  .*      *.           *    :      |
   20000 ++    .*.*.. +    *.. : :    :  *.  +   ..  *.    .*. +     *.     |
         |    *      *        *   : .*        *.*      *..*   *        *..  |
   15000 ++                       *.                                      *.|
         |                                                                  *
   10000 ++-----------------------------------------------------------------+



                           lock_stat.&rq->lock.con-bounces

   75000 ++-----------------------------------------------------------------+
   70000 ++                                   O                             |
         |                      O                                           |
   65000 ++       O                                         O               |
   60000 ++     O             O      O O    O        O          O  O   O    O
   55000 O+O  O      O O O O      O             O  O   O  O          O      |
   50000 ++                              O                    O           O |
         |                                                                  |
   45000 ++                                                                 |
   40000 ++                                                                 |
   35000 ++                                                                 |
   30000 ++                                                                 |
         |            .*.*.  .*.    .*.        .*..                         |
   25000 *+*..*.*.*..*     *.   *.*.   *.*..*.*    *. .*..*.*.*.*..*.*.*..*.*
   20000 ++------------------------------------------*----------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                19932.93       +69.5%     33789.80  lkp-a04/micro/netperf/120s-200%-TCP_RR
                24604.53      +133.9%     57539.47  vpx/micro/hackbench/1600%-process-pipe
                44537.47      +105.1%     91329.27  TOTAL lock_stat.&rq->lock.con-bounces


                          lock_stat.&p->pi_lock.acq-bounces

   65000 ++-----------------------------------------------------------------+
   60000 ++                                                   O             |
         |                                                                  |
   55000 ++                                                                 |
   50000 ++          O                                                      |
   45000 ++     O        O             O                                    |
   40000 ++                     O O                             O           |
         |                                    O        O  O                 *
   35000 O+                              O  O      O O      O      O       :|
   30000 ++O  O        O   *    *    O                                     :|
   25000 ++       O        ::   ::              O         *              .* |
   20000 *+               :  :O: :             .*..      + +         O *. O |
         |+  .*.*.    .*. :  : :  : .*. .*..*.*         +   *.*.*..*. +     |
   15000 ++*.     *..*   *    *   *.   *           *.*.*             *      |
   10000 ++-----------------------------------------------------------------+



                           lock_stat.&p->pi_lock.acq-bounces

   1.1e+07 ++----------------------------------O----------------------------+
           |        O                   O             O            O        |
     1e+07 ++O             O     O                          O           O   |
     9e+06 ++                                                               |
           |                                                                |
     8e+06 ++                                                               |
           |                                                                |
     7e+06 ++                                                               |
           O   O  O   O O    O O    O O      O   O      O O      O   O      O
     6e+06 ++                             O        O           O          O |
     5e+06 ++                                                               |
           |   *..               *               *                          |
     4e+06 *+ +   *.   .*..     + +             + +  .*.*.      .*.    .*.  |
           | *      *.*    *.*.*   + .*.*.*..*.*   *.     *.*..*   *.*.   *.*
     3e+06 ++-----------------------*---------------------------------------+



                           lock_stat.&p->pi_lock.acq-bounces

   2.2e+06 ++---------------------------------------------------------------+
           |                     O           O                          O   |
     2e+06 O+     O O O    O O                 O      O   O O  O O O      O O
           | O O        O      O    O O O O      O      O            O      |
   1.8e+06 ++                                      O                        |
   1.6e+06 ++                                                               |
           |                                                                |
   1.4e+06 ++                                                               |
           |                                                                |
   1.2e+06 ++                                                               |
     1e+06 ++                                                               |
           |.*.      .*.    .*.        .*.*..*.      .*.*.*.                |
    800000 *+  *..*.*   *..*   *.*.. .*        *.*.*.       *..*.*.*.*..*.*.*
           |                        *                                       |
    600000 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                19103.67       +82.3%     34831.38  kbuildx/micro/netperf/120s-200%-TCP_STREAM
              3788432.77       +99.9%   7574236.37  vpx/micro/hackbench/1600%-process-pipe
               866568.43      +122.8%   1931032.73  vpx/micro/hackbench/1600%-threads-socket
              4674104.87      +104.1%   9540100.48  TOTAL lock_stat.&p->pi_lock.acq-bounces


                      lock_stat.&mm->mmap_sem-W.holdtime-total

   1e+07 ++-----------------------------------------------------------------+
   9e+06 ++                              O                                  |
         |                                                                  |
   8e+06 ++                                                                 |
   7e+06 ++                                     *                           |
         |                                      :                           |
   6e+06 ++                                    : :                          |
   5e+06 ++                                    : :                          |
   4e+06 ++                       O            :  :           O    O        |
         *           O O                      O:  :O O O                    O
   3e+06 ++   *   *..    O *..       O  .*..  :    :      O O   O .*        |
   2e+06 ++O + + +O       +   O O    *.*    O.* O  *           .*.  +O O  O |
         | :+   *    *.*.*    *.   ..       *       + .*.. .*.*      *.*.. .*
   1e+06 ++*                    *.*                  *    *               * |
       0 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              1917967.69       +63.5%   3135013.86  vpx/micro/hackbench/1600%-threads-socket
              1917967.69       +63.5%   3135013.86  TOTAL lock_stat.&mm->mmap_sem-W.holdtime-total


                   lock_stat.&rq->lock.contentions.try_to_wake_up

   90000 ++-----------------------------------------------------------------+
         |                                                                  |
   80000 ++   O                                                             |
   70000 ++O                    O               O                  O        |
         |           O     O      O  O   O         O      O O          O    |
   60000 ++       O      O    O             O        O        O           O |
         O      O                             O        O           * O      O
   50000 ++                            O                        O + :       |
         *.*             *.     *      *    *                    +  :       |
   40000 ++ +    .*      : *.. + :    + + .. +    .*.       *. .*    :      |
   30000 ++  + .*  :    :     *  :  .*   *    *.*.   *      : *      *.*..  |
         |    *     :  O:         *.                  :    :              *.|
   20000 ++         : .*                              :    :                *
         |           *                                 *..*                 |
   10000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                33755.80       +83.1%     61813.93  lkp-a04/micro/netperf/120s-200%-TCP_RR
                33755.80       +83.1%     61813.93  TOTAL lock_stat.&rq->lock.contentions.try_to_wake_up


                     lock_stat.&(&base->lock)->rlock.con-bounces

   40000 ++---O-------------------------------------------------------------+
         |                                                                  |
   35000 ++O                    O               O                  O        |
         |                               O                                  |
         |           O     O      O  O      O      O      O O          O    |
   30000 ++       O      O    O                      O        O      O    O |
         O      O      O                      O        O                    O
   25000 ++                            O                           *        |
         |               *      *                               O + :       |
   20000 *+*            + +    ::      *    *      *             +  :       |
         |  +    .*.. .*   *.. : :    : + .. :    + +           *    :      |
         |   + .*    *        *  :    :  *   :   +   *.    .*. +     *.*..  |
   15000 ++   *                   : .*        *.*      *..*   *           *.|
         |                        *.                                        *
   10000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                17352.67       +75.4%     30441.17  lkp-a04/micro/netperf/120s-200%-TCP_RR
                17352.67       +75.4%     30441.17  TOTAL lock_stat.&(&base->lock)->rlock.con-bounces


                     lock_stat.&(&base->lock)->rlock.contentions

   40000 ++---O-------------------------------------------------------------+
         |                                                                  |
   35000 ++O                    O               O                  O        |
         |                               O                                  |
         |           O     O      O  O      O      O      O O          O    |
   30000 ++       O      O    O                      O        O      O    O |
         O      O      O                      O        O                    O
   25000 ++                            O                           *        |
         |               *      *                               O + :       |
   20000 *+*            + +    ::      *    *      *             +  :       |
         |  +    .*.. .*   *.. : :    : + .. :    + +           *    :      |
         |   + .*    *        *  :    :  *   :   +   *.    .*. +     *.*..  |
   15000 ++   *                   : .*        *.*      *..*   *           *.|
         |                        *.                                        *
   10000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                17352.80       +75.4%     30441.47  lkp-a04/micro/netperf/120s-200%-TCP_RR
                17352.80       +75.4%     30441.47  TOTAL lock_stat.&(&base->lock)->rlock.contentions


                         lock_stat.&pipe->mutex/1.acq-bounces

     5e+07 ++---------------------------------------------------------------+
           | O      O                   O      O      O            O        |
   4.5e+07 ++              O     O                                          |
     4e+07 ++                                               O           O   |
           |                                                                |
   3.5e+07 ++                                                               |
           |                                                                |
     3e+07 ++                                                               |
           |      O                                                         |
   2.5e+07 O+  O      O O    O O    O O   O  O   O O    O O    O O   O    O O
     2e+07 ++  *..*     *        *               *    *.         *          |
           *  :    :   + +      : +             : + ..  *       : :     *   |
   1.5e+07 ++ :    : .*   +     :  + .*.        :  *     +      : :   .. + .*
           | *      *      *.*.*    *   *.*..*.*          *.*..*   *.*    * |
     1e+07 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             16086446.83       +96.0%  31532678.93  vpx/micro/hackbench/1600%-process-pipe
             16086446.83       +96.0%  31532678.93  TOTAL lock_stat.&pipe->mutex/1.acq-bounces


                           lock_stat.&pipe->wait.acq-bounces

     5e+07 ++---------------------------------------------------------------+
           | O                          O      O      O            O        |
   4.5e+07 ++       O      O     O                                          |
     4e+07 ++                                               O           O   |
           |                                                                |
   3.5e+07 ++                                                               |
           |                                                                |
     3e+07 ++                                                               |
           |      O                                                         |
   2.5e+07 O+  O      O O    O O    O O   O  O   O O    O O    O O   O    O O
     2e+07 ++  *..*     *        *               *    *.         *          |
           *  :    :   : +      : +             : + ..  *       : :     *   |
   1.5e+07 ++ :    :   :  +     :  + .*.        :  *     +      : :   .. +  |
           | *      *.*    *.*.*    *   *.*..*.*          *.*..*   *.*    *.*
     1e+07 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             16067225.30       +95.8%  31452746.80  vpx/micro/hackbench/1600%-process-pipe
             16067225.30       +95.8%  31452746.80  TOTAL lock_stat.&pipe->wait.acq-bounces


                          lock_stat.&u->readlock.acq-bounces

   650000 ++----------------------------------------------------------------+
          |                                                                 |
   600000 O+O  O          O O      O                 O  O     O  O O        O
          |          O  O        O   O  O O O    O O                 O O    |
   550000 ++       O                          O             O             O |
   500000 ++     O            O                           O                 |
          |                                                                 |
   450000 ++                                                                |
          |                                                                 |
   400000 ++                                           .*                   |
   350000 ++*..           *.*          .*.*         .*.  :        .*.       |
          *    *   *.*.. +   :       *.    + .*.. .*     :   .*..*   *     .*
   300000 ++    + +     *    :  .*  :       *    *        *.*         +  .* |
          |      *            *.  + :                                  *.   |
   250000 ++-----------------------*----------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               330043.57       +71.6%    566403.83  vpx/micro/hackbench/1600%-threads-socket
               330043.57       +71.6%    566403.83  TOTAL lock_stat.&u->readlock.acq-bounces


                            lock_stat.&rq->lock.acq-bounces

   140000 ++---O------------------------------------------------------------+
   130000 ++                                                                |
          | O                    O               O                 O        |
   120000 ++         O      O      O      O        O                        |
   110000 ++       O      O   O      O      O        O    O O O        O    |
          |      O                            O                      O    O |
   100000 O+            O               O               O                   O
    90000 ++                                                     O *        |
    80000 ++                                                      + :       |
          *.*            .*.     *      *   *      *             *  :       |
    70000 ++ +   *.*.  .*   *. .. :   .. + + :    : +       *. ..    :      |
    60000 ++  + +    *.       *   : .*    *  :    :  *..*. +  *      *.*..  |
          |    *                   *          *..*        *               *.|
    50000 ++                                                                *
    40000 ++----------------------------------------------------------------+



                            lock_stat.&rq->lock.acq-bounces

   260000 ++----------------------------------O-----------------------------+
          |                      O                                          |
   240000 ++       O                                                        |
          | O                           O            O      O      O   O    |
   220000 ++              O                                                 |
          |                                                                 |
   200000 ++     O            O      O      O                    O          O
          O    O     O  O   O      O      O      O O    O O          O    O |
   180000 ++                                                  O             |
          *             *                        *     .*              *    |
   160000 ++   *.*     : :       *              : :  *.  :       *     ::   |
          |:  +   :    : :      + :             : : :    :      + :   :  :  |
   140000 ++:+    :   :   :    +  :            :   ::     :    +  :   :  :  |
          | *      *.*    *.*.*    *.*..*.*.*.*    *      *.*.*    *.*    *.*
   120000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                65459.67       +70.0%    111253.97  lkp-a04/micro/netperf/120s-200%-TCP_RR
               141630.53       +45.9%    206622.70  vpx/micro/hackbench/1600%-process-pipe
               207090.20       +53.5%    317876.67  TOTAL lock_stat.&rq->lock.acq-bounces


                         lock_stat.&pipe->wait.waittime-total

   320000 ++----------------------------O-----O-----------------------------+
   300000 ++                                                                |
          |                                                                 |
   280000 *+                                            O                   |
   260000 ++O                    O                               O O   O    O
   240000 ++       O               O O               O               O      |
   220000 ++   O        O O O                             O O               |
          O:                  O                  O                          |
   200000 ++   * O   O                           *                     *    |
   180000 ++:.. +                         O O   + :O          O       : + O |
   160000 ++*    *.     *.      .*   *..       +  :    .*             :  +  |
   140000 ++       *. ..  *. .*.  + +    .*. .*    :.*.  +    *..*. .*    *.*
          |          *      *      *    *   *      *      *. +     *        |
   120000 ++                                                *               |
   100000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               154280.40       +49.5%    230670.34  vpx/micro/hackbench/1600%-process-pipe
               154280.40       +49.5%    230670.34  TOTAL lock_stat.&pipe->wait.waittime-total


                         lock_stat.slock-AF_INET/1.acq-bounces

   1.6e+06 ++---------------------------------------------------------------+
           |                                                                |
   1.4e+06 *+                                         O              O      |
           |                                            O  O                |
   1.2e+06 ++                       O                        O              |
           |:                     O           O                   O    O    |
     1e+06 ++   O  O O    *..               O    *.             *           O
           | :  *..*.*    :  *         O        +  *..          :           |
    800000 O+:  :     + O:   O+O    *          + O             : :          |
           | :O:       + :     *    ::        *    O  *.*..    : :          |
    600000 ++: :        *       +  :  :  O   :             *  :  :        O |
           |  :                  + :  :      :              + :  :          |
    400000 ++ *                   *    *.  .*                *    : .*.*..* |
           |                             *.                       *.        |
    200000 ++---------------------------------------------------------------+



                        lock_stat.slock-AF_INET/1.acq-bounces

   75000 ++-----------------------------------------------------------------+
   70000 ++   O                                                             |
         | O                             O      O                           |
   65000 ++                     O O                         O      O        |
   60000 ++       O  O   O O  O      O      O      O O    O   O        O    |
   55000 ++                                   O        O             O      |
   50000 O+     O      O               O                                  O O
         |                                                      O           |
   45000 ++                                                        *        |
   40000 ++                                                      .. :       |
   35000 *+*..          .*.     *      *.  .*      *.           *   :       |
   30000 ++    .*.*.. .*   *.. + :    +  *.  +   ..  *.    .*. +     *.     |
         |    *      *        *  :  .*        *.*      *..*   *        *..*.|
   25000 ++                       *.                                        *
   20000 ++-----------------------------------------------------------------+



                         lock_stat.slock-AF_INET/1.acq-bounces

   280000 ++----------------------------------------------------------------+
          |                               O                                 |
   270000 ++                  O                           O                 |
          |          O      O      O    O     O  O      O   O O  O   O    O O
   260000 O+   O O O    O        O                 O O                 O    |
          | O             O          O      O                      O        |
   250000 ++                                                                |
          |                                                                 |
   240000 ++                                                                |
          |                                                                 |
   230000 ++  .*.          .*.*..                           *.*..*. .*.     |
          *.*.   *.*.*.. .*      *.*.*..*.*.*.*..*.*.*..*. +       *   *..*.*
   220000 ++            *                                 *                 |
          |                                                                 |
   210000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               671930.42       +47.0%    987764.22  kbuildx/micro/netperf/120s-200%-TCP_MAERTS
                30989.57       +89.5%     58736.60  lkp-a04/micro/netperf/120s-200%-TCP_RR
               225877.87       +16.3%    262641.73  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               928797.86       +41.0%   1309142.56  TOTAL lock_stat.slock-AF_INET/1.acq-bounces


                         lock_stat.&rq->lock.waittime-total

   40000 ++--------------------------O--------------------------------------+
         |    O O                               O  O                        |
   38000 ++            O        O O         O O                    O   O  O O
         |                                                                  |
   36000 ++O                  O          O                O     O           |
         O           O   O             O             O               O      |
   34000 ++       O        O                           O                    |
         |                                                  O             * |
   32000 ++   *                   *                           O          : :|
         |  .. +                  :+        *.             .*            : :|
   30000 ++*    *     .*.*    *  :  + .*. ..  *     .*.  .*  :    .*.*. :   *
         |+      +  .*    + .. + :   *   *     :   *   *.    : .*.     *    |
   28000 *+       *.       *    *              :  +           *             |
         |                                      :+                          |
   26000 ++-------------------------------------*---------------------------+



                          lock_stat.&rq->lock.waittime-total

   400000 ++---O------------------------------------------------------------+
          |                                                        O        |
   350000 ++O                    O               O                          |
          |          O                    O        O                        |
          |        O        O      O O      O        O    O O          O    |
   300000 ++              O   O               O               O      O    O |
          O      O      O               O               O                   O
   250000 ++                                                       *        |
          |                                                      O: :       |
   200000 *+             .*      *      *          *              : :       |
          | *..   .*.  .*  +   .. :    + + .*     : +            *   :      |
          |     .*   *.     *.*   :   +   *  +    :  *..   .*. ..    *.*..  |
   150000 ++   *                   :.*        *..*      *.*   *           *.|
          |                        *                                        *
   100000 ++----------------------------------------------------------------+



                          lock_stat.&rq->lock.waittime-total

   115000 ++------------O---------------------------------------------------+
          |                               O                                 |
          O O                 O                  O                     O  O |
   110000 ++       O        O    O      O     O    O             O O        |
          |                                                 O               |
          |    O                     O               O  O O   O      O      |
   105000 ++     O   O    O        O        O                               O
          |                                                                 |
   100000 ++                    .*   *..*     *..                           |
          *         .*..      *. :   :  :     :            .*    *.    *    |
          |:   *   *     .*   :   : :    :   :   *.*.*.. .*  :  +  *. + :   |
    95000 ++ .. : :     *  : :    : :    :   :          *    : +     *   :  |
          | *   : :        : :     :      :.*                 *          :  |
          |      *          *      *      *                               *.*
    90000 ++----------------------------------------------------------------+



                          lock_stat.&rq->lock.waittime-total

   185000 ++--------------------------------------------------------------O-+
          |                                                                 |
   180000 ++   O                                        O                   |
   175000 ++              O   O             O                               |
          |      O O O  O               O     O             O      O   O    |
   170000 O+O               O        O                           O   O      O
   165000 *+                  *                      O  *     O             |
          |:                  :+ O *      O        O   :: O                 |
   160000 ++                 :  + + :       *.   O     : :   .*..         * |
   155000 ++:      *     .*  :   *  :  .*. +  *..     :  :  *    *.     .. +|
          | *..   : +  .*  +:        *.   *          *   : :       *. .*    *
   150000 ++      :  *.     *                    *  :     ::         *      |
   145000 ++   *.*                                + :     *                 |
          |                                        *                        |
   140000 ++----------------------------------------------------------------+



                          lock_stat.&rq->lock.waittime-total

   160000 ++----------------------------------------------------------------+
          |                                                                 |
   140000 ++     O                                                          O
          *          O        O    O        O    O               O          |
   120000 O+                         O             O                      O |
          |:   O        O        O        O   O         O O   O      O      |
   100000 ++                O                               O               |
          |:                                                           O    |
    80000 ++O      O      O             O            O             O        |
          | :             *.*.*                    *      *.                |
    60000 ++*..    *.*.. +     +   *.*..*.*.*.*.. + :    +  *.      .*    *.*
          |       :     *       + +              *  :  .*     *..  *  + ..  |
    40000 ++   *. :              *                   *.           +    *    |
          |      *                                               *          |
    20000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                29657.07       +22.9%     36448.92  lkp-a04/micro/netperf/120s-200%-TCP_CRR
               171199.99       +79.7%    307634.64  lkp-a04/micro/netperf/120s-200%-TCP_RR
                96362.47       +12.5%    108370.82  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               154316.41       +10.1%    169943.20  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                57888.34       +90.7%    110369.77  vpx/micro/hackbench/1600%-process-pipe
               509424.28       +43.8%    732767.35  TOTAL lock_stat.&rq->lock.waittime-total


                   lock_stat.&(&n->list_lock)->rlock.holdtime-total

   120000 ++----------------------------------------------------------------+
   115000 ++     O                                                          |
          |                 O        O                             O O      O
   110000 O+O  O     O           O            O      O  O        O          |
   105000 ++                  O           O      O O          O             |
          |             O          O    O                   O          O  O |
   100000 ++       O      O                 O             O                 |
    95000 ++                                                                |
    90000 ++                                                                |
          |                               *                                 |
    85000 ++       *                      ::                                |
    80000 *+      : :                    : :  *.. .*.*..*           .*.*..* |
          |+      : :  .*. .*.*..   .*.. :  :+   *       :  *.*.. .*        |
    75000 ++*..*.*   *.   *      *.*    :   *            : +     *          |
    70000 ++----------------------------*-----------------*-----------------+



                  lock_stat.&(&n->list_lock)->rlock.holdtime-total

   50000 ++-----------------------------------------------------------------+
         |                        O    O                                    |
   48000 ++                                     O                           |
   46000 ++       O           O             O          O                  O |
         |             O O      O        O                O                 |
   44000 ++   O O    O               O             O O      O   O  O   O    |
         O O               O                  O               O      O      O
   42000 ++                                                                 |
         |                                                                  |
   40000 ++                                                                 |
   38000 ++                     *    *                                      |
         *.                    + + .. +      .*.      .*..  *.*.  .*.    .*.|
   36000 ++*..*.*.*..*   *.*..*   *    *.*..*   *..*.*     +    *.   *.*.   *
         |            + +                                 *                 |
   34000 ++------------*----------------------------------------------------+



                   lock_stat.&(&n->list_lock)->rlock.holdtime-total

   115000 ++---------------------------------------O------------------------+
   110000 ++                                                                |
          |        O      O               O               O O               |
   105000 ++O        O      O O  O   O  O        O   O        O    O O    O O
   100000 ++   O O                            O                  O          |
          O             O          O                    O                   |
    95000 ++                                O                          O    |
    90000 ++                                                                |
    85000 ++                                                                |
          |                                                                .*
    80000 ++                                                         *    * |
    75000 *+    .*. .*..   .*                      *.*..*.*.*.*..   + + ..  |
          |+  .*   *    *.*  +  .*. .*.. .*.*.    +              *.*   *    |
    70000 ++*.                *.   *    *     *..*                          |
    65000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                77344.05       +38.0%    106711.09  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                36611.85       +23.1%     45056.56  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                73665.06       +39.9%    103034.85  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               187620.96       +35.8%    254802.50  TOTAL lock_stat.&(&n->list_lock)->rlock.holdtime-total


                          lock_stat.((&timer)).acquisitions

   26000 ++-------------O---------------------------------------------------+
         |                         O                                        |
   24000 ++                          O                                      |
   22000 ++        O                              *          O              |
         |    O       O         O       O       O+:            O  O         |
   20000 *+      O *..*.      O                 *  :   O  O         O       |
   18000 ++       :     *  O               O   :   : O                      |
         | :O *.. :      :         *         O :    :  *            *..     O
   16000 ++: +   *        :       + :         :     : : +      *    :  *    |
   14000 ++ *             :  .*. +  :      *. :      ::  +    + :  :    :   |
         O                 *.   *    :   ..  *       *    *..*  :  :   O:   |
   12000 ++                          *..*                        : :     *  |
   10000 ++                                                      ::         |
         |                                                        *         |
    8000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                15403.88       +26.6%     19501.07  kbuildx/micro/netperf/120s-200%-TCP_MAERTS
                15403.88       +26.6%     19501.07  TOTAL lock_stat.((&timer)).acquisitions


                  lock_stat.slock-AF_INET/1.contentions.tcp_v4_rcv

   70000 ++---------------------------------------------------O-------------+
         |                 O                O                               |
   60000 ++O                    O                         O            O    |
         O *                  O          O                                  |
         | :+                                 O                      O    O |
   50000 ++  +  O      O             O                 O        *  O        |
         |:   *                   O        .*.  O  O.*.*    O  + +    .*    |
   40000 *+    :     O   O *..*   *..*.O.*.   *    *    :   *.* O + .*  :   |
         |    O:    .*.*. +    :  :    *      :    :    :   :      *    :   |
   30000 ++     *.*.     *     : :             :  :      : :             :  *
         |                      ::             :  :      : :             : +|
         |                      *              : :        *               * |
   20000 ++                                     ::                          |
         |                                      *                           |
   10000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                37354.43       +35.7%     50698.21  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                37354.43       +35.7%     50698.21  TOTAL lock_stat.slock-AF_INET/1.contentions.tcp_v4_rcv


                        lock_stat.slock-AF_INET/1.con-bounces

   70000 ++---------------------------------------------------O-------------+
         |                 O                O                               |
   60000 ++O                    O                         O            O    |
         O *                  O          O                                  |
         | :+                                 O                      O    O |
   50000 ++  +  O      O             O                 O        *  O        |
         |:   *                   O        .*.  O  O.*.*    O  + +    .*    |
   40000 *+    :     O   O *..*   *..*.O.*.   *    *    :   *.* O + .*  :   |
         |    O:    .*.*. +    :  :    *      :    :    :   :      *    :   |
   30000 ++     *.*.     *     : :             :  :      : :             :  *
         |                      ::             :  :      : :             : +|
         |                      *              : :        *               * |
   20000 ++                                     ::                          |
         |                                      *                           |
   10000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                37339.00       +35.6%     50648.00  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                37339.00       +35.6%     50648.00  TOTAL lock_stat.slock-AF_INET/1.con-bounces


                        lock_stat.slock-AF_INET/1.contentions

   70000 ++---------------------------------------------------O-------------+
         |                 O                O                               |
   60000 ++O                    O                         O            O    |
         O *                  O          O                                  |
         | :+                                 O                      O    O |
   50000 ++  +  O      O             O                 O        *  O        |
         |:   *                   O        .*.  O  O.*.*    O  + +    .*    |
   40000 *+    :     O   O *..*   *..*.O.*.   *    *    :   *.* O + .*  :   |
         |    O:    .*.*. +    :  :    *      :    :    :   :      *    :   |
   30000 ++     *.*.     *     : :             :  :      : :             :  *
         |                      ::             :  :      : :             : +|
         |                      *              : :        *               * |
   20000 ++                                     ::                          |
         |                                      *                           |
   10000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                37339.93       +35.6%     50649.17  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                37339.93       +35.6%     50649.17  TOTAL lock_stat.slock-AF_INET/1.contentions


                     lock_stat.hrtimer_bases.lock#1.holdtime-total

   1.8e+06 ++---------------------------------------------------------------+
           |                                                         O      |
   1.7e+06 ++                                         O       O             |
           O                                   O                            O
           |                                                                *
   1.6e+06 ++                           O                                  :|
           |             O       *..                                      : |
   1.5e+06 ++                   .   ..                                   :  |
           |                   .      ..                                 :  |
   1.4e+06 ++                ..                                         :   |
           |                .           *.....               .*.....   :    |
           |      O        .     O            .            ..       . :     |
   1.3e+06 *+.....*...... .                    *.....    ..          *      |
           |             *                           . ..                   |
   1.2e+06 ++-----------------------------------------*---------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              1351815.26       +18.0%   1595184.74  lkp-snb01/micro/hackbench/1600%-threads-pipe
              1351815.26       +18.0%   1595184.74  TOTAL lock_stat.hrtimer_bases.lock#1.holdtime-total


                lock_stat.((&icsk->icsk_delack_timer)).holdtime-total

   65000 ++--------------------------------------------------------O--------+
         |                    O                                             |
         | O    O                                      O             O      O
   60000 ++   O            O    O        O  O      O            O      O    |
         O           O O O        O  O        O                             |
         |        O                    O        O    O    O O O             |
   55000 ++                                                                 |
         *                                                                O |
   50000 ++                  .*                                             |
         |:               .*.  +                            *.*    *      * |
         | *..  *.     *.*      *.  .*.             .*     +   + .. :   ..  |
   45000 ++    +  *.. +           *.   *.  .*.*.*..*  +  .*     *   : .*    |
         |    *      *                   *.            *.            *      |
         |                                                                  |
   40000 ++-----------------------------------------------------------------+



                lock_stat.((&icsk->icsk_delack_timer)).holdtime-total

   54000 ++-------------------O---------------------------------------------+
   52000 ++                                                                 |
         |           O                                                 O    |
   50000 ++            O          O                O               O        |
   48000 ++O  O   O             O    O O O      O    O      O   O    O    O |
         |                 O                  O        O                    |
   46000 ++     O        O                  O                 O             O
   44000 O+                                               O                 |
   42000 ++                     *                                           |
         |    *     .*         ::        *                                  |
   40000 ++  : :  *.  :    *.. : :      + :   *.      .*..                  |
   38000 ++  : : :    :   +   *  :   *.*  :   : *..*.*    *.*.    .*.    .* |
         *. :   ::     :.*        :..      : :                *.*.   *.*.  +|
   36000 ++*    *      *          *        : :                              *
   34000 ++---------------------------------*-------------------------------+



                lock_stat.((&icsk->icsk_delack_timer)).holdtime-total

   65000 ++-----------------------------------------------------------------O
         |                                         O                        |
   60000 ++              O    O        O      O                             |
         |        O             O    O   O  O        O    O O        O      |
         | O         O O          O             O             O    O      O |
   55000 O+   O O                                      O        O      O    |
         |                 O                                                |
   50000 ++                                        *.                       |
         |                                        :  *.            *.*.    .*
   45000 ++    .*    *.*. .*..*.    .*.    .*     :    *..    *. ..    *..* |
         *.*..*  + ..    *      *.*.   *.*.  :   :        *. +  *           |
         |        *                          :   :          *               |
   40000 ++                                   *.*                           |
         |                                                                  |
   35000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                46648.67       +25.4%     58520.11  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                38080.71       +25.8%     47904.90  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                44714.45       +28.6%     57519.14  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               129443.83       +26.7%    163944.15  TOTAL lock_stat.((&icsk->icsk_delack_timer)).holdtime-total


                      lock_stat.rcu_read_lock_bh-R.holdtime-total

     5e+07 ++---------------------------------------------------------------+
           |                                                                |
   4.5e+07 ++                          *.*..                      *      .* |
           O  *                       :                           ::   *. O |
     4e+07 ++: :                  *   :     *.*                  : :   :    |
           | : :     *           : + :         :      *.  .*     :  : :     |
   3.5e+07 ++   *.. +:           :  *          :    ..  *. : O   :  : :O    |
           *  O    *  :     .*. :               :  *       O:   :    *      O
     3e+07 ++         :   *.   *              O : +   O     :   * O         |
           |           :  :  O         O         * O    O   :  +O           |
   2.5e+07 ++   O    O : :O       O              O           :+             |
           |            ::     O                             *              |
     2e+07 ++      O    *                O                                  |
           |            O                                                   |
   1.5e+07 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             35496870.13       -17.8%  29161829.29  kbuildx/micro/netperf/120s-200%-TCP_MAERTS
             35496870.13       -17.8%  29161829.29  TOTAL lock_stat.rcu_read_lock_bh-R.holdtime-total


                           lock_stat.&wq->wait.holdtime-total

    1.3e+07 ++--------------------------------------------------------------+
            |      *                     *                                  |
   1.25e+07 ++   .. :  *. .*..*. .*.   .. :  *                  .*.*        |
    1.2e+07 ++*.*   : +  *      *   *.*   : + +  .*. .*. .*..*.*    +  .*.* |
            *        *                     *   *.   *   *            *.     |
   1.15e+07 ++                                                              |
            |                                                               |
    1.1e+07 ++                                                              |
            |                                                               |
   1.05e+07 ++       O     O                                 O              |
      1e+07 O+           O        O        O O      O   O O    O        O O |
            | O    O   O      O O   O    O     O  O              O          O
    9.5e+06 ++  O                     O               O            O O      |
            |                                                               |
      9e+06 ++--------------------------------------------------------------+



                           lock_stat.&wq->wait.holdtime-total

    1.8e+07 ++--------------------------------------------------------------+
            |                                                               |
   1.75e+07 ++                            .*.*.           *..               *
            *.*.  .*.*. .*.*.. .*.   .*..*     *..*. .*. +   *. .*.*.  .*. +|
            |   *.     *      *   *.*               *   *      *     *.   * |
    1.7e+07 ++                                                              |
            |                                                               |
   1.65e+07 ++                                                              |
            |                                                               |
    1.6e+07 ++                                                              |
            |                                                               |
            O O    O       O          O        O    O O      O O            O
   1.55e+07 ++  O    O   O    O O O          O    O     O O        O O  O O |
            |          O            O    O O                     O          |
    1.5e+07 ++--------------------------------------------------------------+



                           lock_stat.&wq->wait.holdtime-total

   1.35e+07 ++--------------------------------------------------------------+
            |                           .*       .*                         |
    1.3e+07 ++*.     *. .*.    .*. .*.*.  +   .*.  :            .*.         |
   1.25e+07 ++  *.. +  *   *..*   *        *.*     : .*.*.*.. .*   *.  .*.  |
            *      *                                *        *       *.   *.|
    1.2e+07 ++                                                              *
            |                                                               |
   1.15e+07 ++                                                              |
            |                                                               |
    1.1e+07 ++                               O          O               O   |
   1.05e+07 ++  O        O    O     O          O                            |
            O O    O O O   O                          O        O O   O    O |
      1e+07 ++                  O O   O  O O      O       O  O     O        O
            |                                                               |
    9.5e+06 ++--------------------------------------O-----------------------+



                          lock_stat.&wq->wait.holdtime-total

     6e+07 ++---------------------------------------------------------------+
           |                                                                |
   5.5e+07 ++         O                                        O   O        O
           O      O     O      O             O   O    O                 O   |
     5e+07 ++O             O        O   O                 O                 |
           |                 O        O                 O        O   O      |
   4.5e+07 ++  O                 O        O    O   O        O           *   |
           |      * O          *             *            *             : O |
     4e+07 *+    : :           ::           : :          : :           : :  |
           |:    : :          : :           : :          : :           : :  |
   3.5e+07 ++   :   *.*.*..*. :  *..*.*.   :   *.*.*..*.*   : .*.*.   :   :.*
           | *.*             *          *.*                 *.     *. :   * |
     3e+07 ++                                                        *      |
           |                                                                |
   2.5e+07 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             12186291.25       -18.1%   9975400.69  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
             17306687.49       -10.5%  15482186.33  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
             12687009.77       -18.7%  10310970.48  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             35653899.06       +38.4%  49357031.93  vpx/micro/hackbench/1600%-threads-socket
             77833887.57        +9.4%  85125589.43  TOTAL lock_stat.&wq->wait.holdtime-total


                          lock_stat.&p->pi_lock.holdtime-total

   1.15e+07 ++--------------------------------------------------------------+
            |      *                     *                                  |
            |    .. :  *.*.*..*   *.    + :  *        *   *..   .*.*.       |
    1.1e+07 ++*.*   : :        + +  *. +  : + +  .*. + + +   *.*     *..*.* |
            *        ::         *     *    *   *.   *   *                   |
            |        *                                                      |
   1.05e+07 ++                                                              |
            |                                                               |
      1e+07 ++                                                              |
            |                                                O              |
            |              O                                                |
    9.5e+06 ++       O   O        O        O O      O     O    O        O O |
            O O O      O      O O   O    O     O  O     O                   O
            |      O                  O               O          O   O      |
      9e+06 ++-----------------------------------------------------O--------+



                          lock_stat.&p->pi_lock.holdtime-total

   1.46e+07 ++--------------------------------------------------------------+
   1.44e+07 ++                              .*.           *..               |
            *.*         .*.             .*.*   *..   .*. +   *   *.*.  .*. .*
   1.42e+07 ++ +  .*.*.*   *..*.*.*.*.*.          *.*   *     + +    *.   * |
    1.4e+07 ++  *.                                             *            |
            |                                                               |
   1.38e+07 ++                                                              |
   1.36e+07 ++                                                              |
   1.34e+07 ++                                                              |
            |                                                               |
   1.32e+07 ++                                                              |
    1.3e+07 ++     O                  O             O O                     O
            O O      O   O O  O O O          O O          O  O O   O O    O |
   1.28e+07 ++  O                        O        O     O        O      O   |
   1.26e+07 ++---------O------------O------O--------------------------------+



                          lock_stat.&p->pi_lock.holdtime-total

    1.2e+07 ++--------------------------------------------------------------+
            |                           .*       .*                         |
   1.15e+07 ++*.        .*.    .*. .*.*.  +   .*.  :                        |
            |+  *.. .*.*   *..*   *        *.*     : .*.*.*.. .*.*.*    *.  |
            *      *                                *        *      + ..  *.*
    1.1e+07 ++                                                       *      |
            |                                                               |
   1.05e+07 ++                                                              |
            |                                                               |
      1e+07 ++                               O          O               O   |
            |   O  O     O    O     O          O                            |
            O O      O O   O    O     O               O   O    O O O O    O |
    9.5e+06 ++                    O      O O      O          O              O
            |                                       O                       |
      9e+06 ++--------------------------------------------------------------+



                          lock_stat.&p->pi_lock.holdtime-total

    2.8e+08 ++------------------------O-------------------------------------+
   2.75e+08 O+                                 O  O              O          O
            | O O  O     O O  O O   O        O      O   O O        O        |
    2.7e+08 ++       O            O      O            O        O     O    O |
   2.65e+08 ++         O                   O                                |
            |                                                               |
    2.6e+08 ++                                                              |
   2.55e+08 ++                        *..           *                       |
    2.5e+08 ++        .*   *..*.*    +              :     *..*.*        O  .*
            | *     .*  : :      :  *    *.*.*.*.. : :    :     :  *.*    * |
   2.45e+08 ++::   *    : :      : +               : :   :      : +   +  :  |
    2.4e+08 ++ : ..      *        *               *   :  :       *     + :  |
            |:  *                                     *.*               *   |
   2.35e+08 ++                                                              |
    2.3e+08 *+--------------------------------------------------------------+



                         lock_stat.&p->pi_lock.holdtime-total

   4.5e+07 ++---------------------------------------------------------------+
           |                                                   O   O        |
           |          O                          O    O                     O
     4e+07 O+O          O           O   O                                   |
           |      O        O O               O          O        O      O   |
           |                   O      O                   O          O      |
   3.5e+07 ++  O                          O    O   O        O               |
           |        O            O                                        O |
     3e+07 ++                                                               |
           *      *            *             *            *             *   |
           |:    + +          : :           + +          + :           : :  |
   2.5e+07 ++   +   *.    .*. : :    .*.   +   *. .*..*.*  :  .*.*.    : : .*
           | *.*      *.*.   *   *..*   *.*      *          *.     *. :   * |
           |                                                         *      |
     2e+07 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             11008025.90       -14.3%   9428837.12  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
             14235185.18        -9.8%  12845924.76  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
             11370435.14       -15.1%   9656886.55  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
            246121063.23       +10.3% 271415953.35  vpx/micro/hackbench/1600%-process-pipe
             25062730.17       +50.4%  37694959.87  vpx/micro/hackbench/1600%-threads-socket
            307797439.61       +10.8% 341042561.65  TOTAL lock_stat.&p->pi_lock.holdtime-total


                           lock_stat.&wq->wait.acquisitions

   640000 ++----------------------------*-----------------------------------+
          |     .*   *..               + :  *                               |
   620000 ++  .*  : :   *.*.*.  .*.*. +  : : :   *. .*..  *. .*..*.*.    .* |
   600000 *+*.    : :         *.     *    :: : ..  *     +  *        *.*.   |
          |        *                      *   *         *                   |
   580000 ++                                                                |
   560000 ++                                                                |
          |                                                                 |
   540000 ++                                                                |
   520000 ++              O                                 O               |
          |        O                        O             O   O        O    |
   500000 O+            O        O O      O   O  O O    O                 O |
   480000 ++O  O O   O      O O         O            O               O      O
          |                          O                           O          |
   460000 ++-------------------------------------------------------O--------+



                            lock_stat.&wq->wait.acquisitions

    1.1e+06 ++--------------------------------------------------------------+
            | *                           .*.*.*..       .*..               *
   1.08e+06 ++ +  .*.*.*.*.*..*.   .*.  .*        *.*.*.*    *. .*.*.  .*. +|
   1.06e+06 *+  *.              *.*   *.                       *     *.   * |
            |                                                               |
   1.04e+06 ++                                                              |
   1.02e+06 ++                                                              |
            |                                                               |
      1e+06 ++                                                              |
     980000 ++                                                              |
            O   O  O       O          O        O    O O   O  O O   O O      O
     960000 ++O      O   O    O              O    O     O               O   |
     940000 ++         O        O O O    O O                     O        O |
            |                                                               |
     920000 ++--------------------------------------------------------------+



                           lock_stat.&wq->wait.acquisitions

   680000 ++----------------------------------------------------------------+
   660000 ++*..         *          *    *       .*                          |
          |:          .. +   .*.. + + .. +   .*.  :             .*.    *..  |
   640000 ++   *. .*.*    *.*    *   *    *.*     : .*..*.   .*.   *  :     |
   620000 *+     *                                 *      *.*       + :   *.|
          |                                                          *      *
   600000 ++                                                                |
   580000 ++                                                                |
   560000 ++                                                                |
          |                                 O           O                   |
   540000 ++   O                   O          O                        O    |
   520000 O+     O O O  O   O                                               |
          | O             O   O         O        O   O    O   O  O O O    O |
   500000 ++                     O   O    O                 O               O
   480000 ++---------------------------------------O------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               609296.66       -18.9%    494076.90  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
              1073209.47       -10.5%    960960.67  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               639650.47       -19.6%    514574.77  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              2322156.59       -15.2%   1969612.33  TOTAL lock_stat.&wq->wait.acquisitions


                          lock_stat.slab_mutex.holdtime-total

   140000 ++--------------------------------*-------------------------------+
          |                                 ::            *                 |
   130000 ++                  *..           ::            :            *    |
          |      *.     *     :            :  *    *      ::           ::   |
          |      : *. ..:    :   *. .*     :   +  ::     : :  *..*    :  :  *
   120000 *+    :    *  O:   :     *  : O  :    + : :    : : :   :    :  : +|
          |     :O   O   :   :O        :   :     *  :    :  ::    :   :   * |
   110000 ++    :         :  :     O   :  :          O   :  *     :O :      |
          O:   :          *.:           *.:          *..: O   O    :.* O    |
   100000 ++O .*            *        O    * O      O    *          * O      O
          | *. O          O O                               O    O        O |
          |        O                          O                             |
    90000 ++                                                                |
          |                      O                                          |
    80000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               116723.61       -11.3%    103521.04  vpx/micro/hackbench/1600%-threads-socket
               116723.61       -11.3%    103521.04  TOTAL lock_stat.slab_mutex.holdtime-total


                            lock_stat.&rq->lock.acquisitions

   1.95e+06 ++--------------------------------------------------------------+
            |      *                                                        |
    1.9e+06 ++   .. :  *.                *   *            *..     .*        |
   1.85e+06 ++*.*   : :  *.*..*.*.*.*. .. : : :   *. .*. +   *.*.*  +    .* |
            *        ::               *   : : : ..  *   *            *..*   |
    1.8e+06 ++       *                     *   *                            |
   1.75e+06 ++                                                              |
            |                                                               |
    1.7e+06 ++                                                              |
   1.65e+06 ++                                               O              |
            |        O     O      O                       O             O   |
    1.6e+06 ++           O                 O O    O O          O          O O
   1.55e+06 O+O O  O   O      O O   O    O     O      O O            O      |
            |                         O                          O          |
    1.5e+06 ++-----------------------------------------------------O--------+



                            lock_stat.&rq->lock.acquisitions

      2e+06 ++---------------------------*--------*-------------------------+
            | *          *         .*. .. +     .. :             *.         |
   1.95e+06 ++ +    .*. + +    .*.*   *    *. .*   :  *.*       +  *    *   |
    1.9e+06 ++  *..*   *   *..*              *      :+   +    .*    :  + +  |
            *                                       *     *..*      : +   *.|
   1.85e+06 ++                                                       *      *
    1.8e+06 ++                                                              |
            |                                                               |
   1.75e+06 ++                                                              |
    1.7e+06 ++                               O          O                   |
            |                       O          O                        O   |
   1.65e+06 O+  O  O O O O    O                                             |
    1.6e+06 ++O            O    O O      O        O   O   O    O O   O    O |
            |                         O    O                 O     O        O
   1.55e+06 ++--------------------------------------O-----------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              1848833.76       -14.5%   1580555.97  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
              1921780.70       -15.4%   1625918.30  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              3770614.46       -15.0%   3206474.27  TOTAL lock_stat.&rq->lock.acquisitions


                         lock_stat.&rq->lock/1.holdtime-total

   270000 ++---------O------------------------------------------------------+
          |                               O                                 |
   260000 O+O      O             O            O                      O      |
   250000 ++                            O                                   |
          |      O        O        O               O      O O             O |
   240000 ++   O        O     O                                  O     O    |
          |                 O        O               O      *      O        O
   230000 ++                                O    O      O   : O             |
          |                          *..*        *.        : :              |
   220000 ++       *                 :  :       +  *       : :      .*.*..  |
   210000 ++*..*. +:    *.*      *  :    :     +    :      :  :    *        |
          *      *  :  :   :    + : :    :   .*     :      :  *    :      * |
   200000 ++        :  :   :   +  : :     :.*        *..  :    +  :         |
          |          ::     *.*    :      *             *.*     + :         |
   190000 ++---------*-------------*-----------------------------*----------+



                         lock_stat.&rq->lock/1.holdtime-total

   190000 ++----------------------------------------------------------------+
          |                                                        O   O    |
   180000 ++       O               O             O               O          |
          |                                                          O      |
   170000 O+              O      O   O                                      |
          |          O  O               O O   O    O O    O   O             O
   160000 ++   *     *      O O                                           O |
          | O :O+O  + :                         .*          O        *      |
   150000 ++  :  *.*  :          *          O *.  :     *        *. + +     |
          |  :         :  *    .. :  *..      :   :   .. :      :  *   *..  |
   140000 *+*          : + + .*   : :   *    :     :.*   :      :           |
          |             *   *      ::    +   :     *      *.*. :          *.*
   130000 ++                       *      *.*                 *             |
          |                                                                 |
   120000 ++----------------------------------------------------------------+



                         lock_stat.&rq->lock/1.holdtime-total

   270000 ++----------------------------------------------------------------+
          |                          O             O                        O
   260000 ++              O               O O O             O               |
   250000 ++                            O                 O          O      |
          |        O               O                                   O  O |
   240000 ++   O     O                             *                        |
   230000 ++O    *      O                          : O  O     O  O O        |
          |     + :     *     *..O               O ::            *   *      |
   220000 O+  .* O:    : :  O :  *.*.*..          : :     *     : : : :     |
   210000 *+*.     *   : :   :          *         : :    : +    : : : :     |
          |         : :   *. :           :    *   :  :   :  *. :   *   *..*.*
   200000 ++        : :     *            :   : +  :  *..*     *             |
   190000 ++         *                    *. :  +:                          |
          |                                 *    *                          |
   180000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               209008.67       +17.5%    245514.22  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               142473.68       +17.0%    166643.24  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               210113.45       +15.0%    241659.81  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               561595.80       +16.4%    653817.27  TOTAL lock_stat.&rq->lock/1.holdtime-total


                          lock_stat.&p->pi_lock.acquisitions

   400000 ++----------------------------------------------------------------+
   390000 ++     *                      *                                   |
          |     + :  *..*.             + :  *             *. .*..           |
   380000 ++*..*  : :     *.*.*..*.*. +  : : +  .*. .*.. +  *    *.*. .*..* |
   370000 *+       ::                *    ::  *.   *    *            *      |
          |        *                      *                                 |
   360000 ++                                                                |
   350000 ++                                                                |
   340000 ++                                                                |
          |                                                 O               |
   330000 ++       O    O O      O          O             O   O        O    |
   320000 O+                       O      O   O  O O                      O O
          | O  O O   O      O O         O            O  O            O      |
   310000 ++                         O                           O O        |
   300000 ++----------------------------------------------------------------+



                          lock_stat.&p->pi_lock.acquisitions

   630000 ++----------------------------------------------------------------+
          |                                                                 |
   620000 ++*..        .*.              *.*.*.*.. .*.  .*.*.     *.   .*..  *
   610000 *+   *.*.*.*.   *.*.*..*.*. ..         *   *.     *. ..  *.*     +|
          |                          *                        *           * |
   600000 ++                                                                |
   590000 ++                                                                |
          |                                                                 |
   580000 ++                                                                |
   570000 ++                                                                |
          |                                          O                      O
   560000 O+O  O O        O O        O        O    O      O O O    O O      |
   550000 ++       O    O     O  O O        O    O      O              O  O |
          |          O                  O O                      O          |
   540000 ++----------------------------------------------------------------+



                          lock_stat.&p->pi_lock.acquisitions

   410000 ++*---------------------------*--------*--------------------------+
   400000 ++ +         .*.   .*.. .*. .. +   .*.  :             .*.         |
          |:  +   .*.*.   *.*    *   *    *.*     : .*..*.   .*.   *   *..  |
   390000 *+   *.*                                 *      *.*       + +     |
   380000 ++                                                         *    *.*
          |                                                                 |
   370000 ++                                                                |
   360000 ++                                                                |
   350000 ++                                                                |
          |                                 O           O                   |
   340000 O+   O        O   O      O          O                        O    |
   330000 ++O    O O O    O                          O           O   O    O |
          |                   O  O   O  O O      O        O O O    O        O
   320000 ++                                       O                        |
   310000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               378244.76       -15.3%    320214.60  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               612947.03        -9.3%    555863.47  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
               394025.13       -16.2%    330323.63  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1385216.93       -12.9%   1206401.70  TOTAL lock_stat.&p->pi_lock.acquisitions


                        lock_stat.slock-AF_INET.waittime-total

   5.4e+06 ++---------------------------------------------------------------+
   5.3e+06 ++         O                   O                                 |
           |                            O                        O          |
   5.2e+06 ++                O O               O        O O          O      |
   5.1e+06 ++  O  O O   O           O            O O        O  O        O O O
     5e+06 O+O             O     O    O               O            O        |
   4.9e+06 ++                                O                              |
           |                                                                |
   4.8e+06 ++                                                               |
   4.7e+06 ++                                                               |
   4.6e+06 ++                                                               |
   4.5e+06 ++  *..          .*.                               .*.           |
           |  +    .*.*    *   *.*.. .*. .*..*. .*.*.. .*. .*.   *. .*.. .*.|
   4.4e+06 *+*    *    + ..         *   *      *      *   *        *    *   *
   4.3e+06 ++-----------*---------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              4433482.45       +15.0%   5099209.90  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
              4433482.45       +15.0%   5099209.90  TOTAL lock_stat.slock-AF_INET.waittime-total


                      lock_stat.&(&zone->lock)->rlock.contentions

   116000 ++-----------------------------------------------------O----------+
   114000 ++   O O                   O                               O      O
          |          O      O O           O   O  O O O  O          O        |
   112000 O+O                    O O        O                 O        O    |
   110000 ++       O    O O             O                   O             O |
          |                                               O                 |
   108000 ++                                                                |
   106000 ++                                                                |
   104000 ++                                                                |
          |                                                                 |
   102000 *+       *                      *        *    *                   |
   100000 ++       :+                *.. : :      + + .. +   .*..     .*..  |
          |:  .*  :  *..*.*. .*..*. +    : : .*..*   *    *.*     .*.*    * |
    98000 ++*.  + :         *      *    *   *                    *          |
    96000 ++-----*----------------------------------------------------------+



                      lock_stat.&(&zone->lock)->rlock.contentions

   115000 ++----------------------------------------------------------------+
          |                                                                 |
          |                               O        O      O                 |
   110000 ++                  O  O   O           O   O      O      O      O O
          | O  O   O      O             O                     O  O   O      |
          O      O   O             O                    O              O    |
   105000 ++            O   O               O O                             |
          |                                                                 |
   100000 ++                                                                |
          |                                                                .*
          |                                        *.               .*    * |
    95000 *+  .*.   .*..             *..   .*.    :  *..*.*.    .*.*  + ..  |
          | *.   *.*    *.*.*.  .*. +    .*   *.. :         *.*.       *    |
          |                   *.   *    *        *                          |
    90000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                99108.17       +13.5%    112503.63  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                94448.70       +14.2%    107816.80  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               193556.87       +13.8%    220320.43  TOTAL lock_stat.&(&zone->lock)->rlock.contentions


                      lock_stat.&(&zone->lock)->rlock.con-bounces

   116000 ++-----------------------------------------------------O----------+
   114000 ++   O O                   O                               O      O
          |          O      O O           O   O  O O O  O          O        |
   112000 O+O                    O O        O                 O        O    |
   110000 ++       O    O O             O                   O             O |
          |                                               O                 |
   108000 ++                                                                |
   106000 ++                                                                |
   104000 ++                                                                |
          |                                                                 |
   102000 *+       *                      *             *                   |
   100000 ++       :+                *.. : :      .*. .. +   .*..     .*..  |
          |:  .*  :  *..*.*. .*..*. +    : : .*..*   *    *.*     .*.*    * |
    98000 ++*.  + :         *      *    *   *                    *          |
    96000 ++-----*----------------------------------------------------------+



                      lock_stat.&(&zone->lock)->rlock.con-bounces

   115000 ++----------------------------------------------------------------+
          |                                                                 |
          |                                        O      O                 |
   110000 ++                  O  O   O    O      O   O             O      O O
          | O  O   O      O             O                   O O  O   O      |
          O      O   O             O                    O              O    |
   105000 ++            O   O               O O                             |
          |                                                                 |
   100000 ++                                                                |
          |                                                                .*
          |                                                         .*    * |
    95000 *+  .*.   .*..             *..   .*.     *.*..*.*.    .*.*  + ..  |
          | *.   *.*    *.*.*.  .*. +    .*   *.. +         *.*.       *    |
          |                   *.   *    *        *                          |
    90000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                99093.03       +13.5%    112480.70  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                94434.00       +14.1%    107794.90  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               193527.03       +13.8%    220275.60  TOTAL lock_stat.&(&zone->lock)->rlock.con-bounces


                      lock_stat.hrtimer_bases.lock#2.acquisitions

   4.4e+06 ++----------------------------------*----------------------------+
   4.3e+06 ++                           *...    :                           |
           |      O   ...*             :         :            *......*      |
   4.2e+06 ++     *...    :            :          :         ..        +     |
   4.1e+06 ++    .         :          :            :       .           +    |
           |    .          :         :             :      .             +   |
     4e+06 ++ ..            :        :              :   ..               +  |
   3.9e+06 ++.               :      :                : .                  + |
   3.8e+06 O+                 :  O  :                 *                    +|
           *                   :   :                  O              O      *
   3.7e+06 ++                  :  :                           O             O
   3.6e+06 ++                   : :                                         |
           |                     *             O                            |
   3.5e+06 ++            O              O                                   |
   3.4e+06 ++---------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              4046619.50        -7.5%   3741545.80  lkp-snb01/micro/hackbench/1600%-threads-pipe
              4046619.50        -7.5%   3741545.80  TOTAL lock_stat.hrtimer_bases.lock#2.acquisitions


                      lock_stat.slock-AF_INET/1.waittime-total

   44000 ++-----------------------------------------------------------------+
         |                                           O                      |
   42000 ++                                                                 O
   40000 ++                                                                 |
         O                                    O    O                        |
   38000 ++     O O    O             O   O                                O |
         |                        O             O           O        O      |
   36000 ++*         O   O           * O               O  O                 |
         | O:              O  O.*   + :     O               *   O      O    |
   34000 ++ : O      *. .*.*..*  + +  :     *.*.*          : :O   .*. .*..  |
   32000 ++  :     ..  *          *    *   +     +         : :  *.   *      |
         *   :   .*                     + +       + .*.*..*   :+          * |
   30000 ++   *.*                        *         *          *             |
         |                                                                  |
   28000 ++-----------------------------------------------------------------+



                      lock_stat.slock-AF_INET/1.waittime-total

   95000 ++-----------------------------------------------------------------+
         |                               O                                  |
   90000 ++                                                                 |
         |           O     O  O   O    O      O           O   O O      O    |
         |    O   O    O O           O          O    O O             O      O
   85000 O+O    O               O                  O               O      O |
         |                                  O               O               |
   80000 ++                                                                 |
         |    *                .*                           *               |
   75000 *+  + + .*..     .*..*  +  .*. .*..   .*..*.*.*    :+      .*.  .* |
         |+ +   *    *. .*        *.   *    *.*         +  :  *.*..*   *.  +|
         | *           *                                 + :                *
   70000 ++                                               *                 |
         |                                                                  |
   65000 ++-----------------------------------------------------------------+



                      lock_stat.slock-AF_INET/1.waittime-total

   42000 ++-----------------------------------------------------------------+
         |                                                                  |
   40000 ++                       O  O                          O           |
         |                                                                  |
   38000 ++       O  O   O                    O                             |
         |    O               O O      O        O      O             O      |
   36000 O+O    O      O                 *..       O O                      O
         *.                             :   O               O O        O  O |
   34000 ++*..*   *..      O            :   *             O  .*.*..         |
         |     :  :      *.  .*.*.*..  *     :    .*   *.. .*      *.       |
   32000 ++    : :   *. +  *.         +      :  *.  + +   *          *.*..*.*
         |      ::     *             *        :+     *                      |
   30000 ++     *                             *                             |
         |                                                                  |
   28000 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                32619.84       +13.1%     36892.00  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                74441.51       +16.4%     86639.91  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                32598.34       +11.6%     36387.03  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               139659.69       +14.5%    159918.93  TOTAL lock_stat.slock-AF_INET/1.waittime-total


                          lock_stat.slock-AF_INET.con-bounces

   106000 ++------------------------------O---------------------------------+
   104000 ++                                                                |
          |                   O         O     O         O O                 |
   102000 ++         O  O   O                    O O          O  O        O O
   100000 O+     O O             O O                 O      O        O O    |
          | O  O          O                 O                      O        |
    98000 ++                         O                                      |
    96000 ++                                                                |
    94000 ++                                                                |
          |                                                                 |
    92000 ++   *           .*.                              *.*..           |
    90000 ++ .. +         *   *..*. .*.. .*.*.  .*.*.  .*  :     *. .*.*..*.|
          *.*    *.*.*.. +         *    *     *.     *.  + :       *        *
    88000 ++            *                                 *                 |
    86000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                89783.57       +12.6%    101075.83  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                89783.57       +12.6%    101075.83  TOTAL lock_stat.slock-AF_INET.con-bounces


                    lock_stat.slock-AF_INET.contentions.tcp_v4_rcv

   106000 ++------------------------------O---------------------------------+
   104000 ++                                                                |
          |                   O         O     O         O O                 |
   102000 ++         O  O   O                    O O          O  O        O O
   100000 O+     O O             O O                 O      O        O O    |
          | O  O          O                 O                      O        |
    98000 ++                         O                                      |
    96000 ++                                                                |
    94000 ++                                                                |
          |                                                                 |
    92000 ++   *           .*.                              *.*..           |
    90000 ++ .. +         *   *..*. .*.. .*.*.  .*.*.  .*  :     *. .*.*..*.|
          *.*    *.*.*.. +         *    *     *.     *.  + :       *        *
    88000 ++            *                                 *                 |
    86000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                89780.03       +12.6%    101069.37  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                89780.03       +12.6%    101069.37  TOTAL lock_stat.slock-AF_INET.contentions.tcp_v4_rcv


                          lock_stat.slock-AF_INET.contentions

   106000 ++------------------------------O---------------------------------+
   104000 ++                                                                |
          |                   O         O     O         O O                 |
   102000 ++         O  O   O                    O O          O  O        O O
   100000 O+     O O             O O                 O      O        O O    |
          | O  O          O                 O                      O        |
    98000 ++                         O                                      |
    96000 ++                                                                |
    94000 ++                                                                |
          |                                                                 |
    92000 ++   *           .*.                              *.*..           |
    90000 ++ .. +         *   *..*. .*.. .*.*.  .*.*.  .*  :     *. .*.*..*.|
          *.*    *.*.*.. +         *    *     *.     *.  + :       *        *
    88000 ++            *                                 *                 |
    86000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                89784.97       +12.6%    101077.50  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                89784.97       +12.6%    101077.50  TOTAL lock_stat.slock-AF_INET.contentions


                           lock_stat.&rq->lock.holdtime-total

    1.8e+07 ++--------------------------------------------------------------+
            |     .*      .*..    *      *                                  |
   1.75e+07 ++ .*.  :  *.*       : +    + :  *            *..*. .*.*.       |
            |.*     : :       *. :  *. +  : : +  .*. .*. +     *     *.. .* |
            *        ::         *     *    ::  *.   *   *               *   |
    1.7e+07 ++       *                     *                                |
            |                                                               |
   1.65e+07 ++                                                              |
            |                                                               |
    1.6e+07 ++                                               O              |
            |        O   O O      O                 O                   O O |
            O                       O      O O            O    O            |
   1.55e+07 ++O O             O O        O     O  O   O O            O      O
            |      O   O              O                          O          |
    1.5e+07 ++-----------------------------------------------------O--------+



                           lock_stat.&rq->lock.holdtime-total

   1.85e+07 ++---------------------------*--------*-------------------------+
            | *.         *      *. .*.*.  +     .. :              .*        |
    1.8e+07 ++  *..     + +    +  *        *. .*   :   .*.      .*  :   *.  |
            *      *.*.*   *..*              *      :.*   *..*.*    : ..  *.|
   1.75e+07 ++                                      *                *      *
            |                                                               |
    1.7e+07 ++                                                              |
            |                                                               |
   1.65e+07 ++                               O          O                   |
            |                                  O                        O   |
    1.6e+07 ++  O  O O        O     O                                       |
            O O        O O O    O     O  O O          O   O    O O O O    O |
   1.55e+07 ++                    O               O O        O              O
            |                                                               |
    1.5e+07 ++--------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             17385317.35       -10.5%  15557132.21  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
             17917887.06       -11.6%  15841284.13  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
             35303204.41       -11.1%  31398416.35  TOTAL lock_stat.&rq->lock.holdtime-total


          lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist

   116000 ++-----------------------------------------------------O----------+
   114000 ++     O                   O                               O      |
          | O  O     O        O  O        O   O  O O O  O          O   O    O
   112000 O+                O      O        O                 O             |
   110000 ++       O    O O                                 O             O |
   108000 ++                            O                 O                 |
   106000 ++                                                                |
          |                                                                 |
   104000 ++                                                                |
   102000 ++       *                                                        |
   100000 *+       ::                *    *            .*                   |
    98000 ++      : :  .*.*. .*..   : +  : + .*..*.*.*.  + .*.*.. .*. .*..* |
          | *..*. :  *.     *    *. :  + :  *             *      *   *      |
    96000 ++     *                 *    *                                   |
    94000 ++----------------------------------------------------------------+



          lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist

   115000 ++----------------------------------------------------------------+
          |                                        O                        |
          |                   O  O        O               O                 |
   110000 ++                         O           O   O      O      O        O
          | O  O   O O    O             O                     O  O   O    O |
          O      O      O          O          O         O              O    |
   105000 ++                O               O                               |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 |
          |                                        *                 *    *.*
    95000 *+    .*.                                :+               + :  +  |
          |+  .*   *.*.. .*.    .*. .*.. .*.*.    :  *..*.*.*.  .*.*  : +   |
          | *.          *   *.*.   *    *     *.. :           *.       *    |
    90000 ++-------------------------------------*--------------------------+



          lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist

   380000 ++----------------------------------------------------------------+
   360000 *+                    O              O      O      O              |
          |:     O       *             O   ....*......*                     O
   340000 O+            : :            *...            :             *      |
   320000 ++:           : :           :                :            : :     |
   300000 ++:          :   :          :                 :          :  :     |
   280000 ++ :        :     :        :                   :         :   :    |
          |   :      :      :       :                    :        :     :   |
   260000 ++  :      :       :      :                     :      :       :  |
   240000 ++   :    :        :     :                      :     :        :  |
   220000 ++   :   :          :   :                        :   :          : |
   200000 ++    : :            : :                          :  :           :|
          |     : :            : :                          : :            :|
   180000 ++     *              *                            *              *
   160000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                98261.38       +14.1%    112089.00  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                93489.13       +15.5%    107966.47  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               274589.80       +27.6%    350468.80  lkp-snb01/micro/hackbench/1600%-process-socket
               466340.31       +22.3%    570524.27  TOTAL lock_stat.&(&zone->lock)->rlock.contentions.get_page_from_freelist


                  lock_stat.tcp_death_row.death_lock.waittime-total

   44000 ++-----------------------------------------------------------------+
         |                               *..                                |
   42000 ++                              :                      *           |
         |                      *    *  :   *   *..*.*          ::          |
   40000 ++                    ::    :+ :   :   :    :         :  :         |
         |                     : :  :  *     : :      :        :  :         |
   38000 ++       O          .*  :  :        : :      :     *.*    *      * |
         |        *      *.*.    :  :        : :   O  :     :       +    + +|
   36000 ++      : :    +        : :          :        :   :         *. +   *
         O.*..  O:  :  *          ::   O      *        *   :  O      O *    |
   34000 *+O  *.*   :O+  O    O   :           O         + O:       O        |
         |    O      *     O      *  O               O O +:     O         O |
   32000 ++                     O                         *                 O
         |             O                 O  O               O          O    |
   30000 ++-------------------------------------O---------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                37180.93       -10.4%     33313.48  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                37180.93       -10.4%     33313.48  TOTAL lock_stat.tcp_death_row.death_lock.waittime-total


                    lock_stat.&(&zone->lock)->rlock.waittime-total

   720000 ++----------------------------------------------------------------+
          |                                                                 O
   700000 ++   O     O  O                          O               O        |
          |                          O               O           O     O    |
   680000 O+O    O O          O  O      O O O O  O      O     O      O      |
          |                 O      O                        O               |
   660000 ++              O                               O               O |
          |                                                                 |
   640000 ++                                                                |
          *.                              *        *    *                   |
   620000 ++*..    *.  .*. .*.*..    *.. + +      : + .. :  *         .*..  |
          |    *. +  *.   *      *. +   *   *.*.. :  *   : + +  .*. .*      |
   600000 ++     *                 *             *        *   *.   *      * |
          |                                                                 |
   580000 ++----------------------------------------------------------------+



                    lock_stat.&(&zone->lock)->rlock.waittime-total

   700000 ++----------------------------------------------------------------+
          |                                                                 |
   680000 ++                         O             O                        |
          |        O          O           O               O          O      O
   660000 ++O               O    O               O          O O  O O      O |
          |      O      O          O    O     O      O                      |
   640000 O+   O          O                             O                   |
          |          O                                                 O    |
   620000 ++                                O                              .*
          |                                                      *   *    * |
   600000 *+        .*               *..                   .*   + :  ::  +  |
          |+  .*.*.*  +    .*    *. +   *. .*.     *.*..*.*  + +  : : : +   |
   580000 ++*.         + .*  + ..  *      *   *.. +           *    ::  *    |
          |             *     *                  *                 *        |
   560000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
               612913.65       +11.2%    681830.33  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
               589188.07       +11.2%    655424.09  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
              1202101.72       +11.2%   1337254.43  TOTAL lock_stat.&(&zone->lock)->rlock.waittime-total


              lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok

   116000 ++---O------------------------------------------------------------+
   114000 ++     O                   O                           O O        |
          |          O      O                 O    O O  O            O      O
   112000 O+O      O    O     O  O        O O    O            O        O    |
   110000 ++              O        O    O                   O             O |
          |                                               O                 |
   108000 ++                                                                |
   106000 ++                                                                |
   104000 ++                                                                |
          |                                                                 |
   102000 *+       *                 *    *        *.  .*                   |
   100000 ++  .*   :+               + +  : :      +  *.  +   .*..     .*..* |
          | *.  : :  *..*.*. .*..*.*   + : : .*..*        *.*    *.*.*      |
    98000 ++    : :         *           *   *                               |
    96000 ++-----*----------------------------------------------------------+



              lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok

   115000 ++----------------------------------------------------------------+
          |                                                                 |
          |                                        O      O                 |
   110000 ++                  O  O        O      O                        O |
          |        O                 O  O            O      O O    O        O
          O O  O O        O                                      O   O O    |
   105000 ++         O  O   O      O        O O         O                   |
          |                                                                 |
   100000 ++                                                                |
          |                                                                 *
          |                          *..   .*.         .*.      .*.        +|
    95000 *+  .*. .*.*.. .*.        +    .*   *..  *.*.   *.*.*.   *.*.  .* |
          | *.   *      *   *.*.. .*    *         +                    *.   |
          |                      *               *                          |
    90000 ++----------------------------------------------------------------+



              lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok

   180000 ++----------------------------------------------------------------+
          |        *                                    *                   |
   175000 ++       ::                                 ..:            *      |
          | *      ::                                *   :       *  : :     |
          | :+    :  *..             *..*.*         :    :  *   + + : :  .*.*
   170000 ++  +   :                 +      +        :    : : + +   *   *.   |
          |:   *  :     *. .*.*.. .*        *.*.. .*      ::  *  O O        |
   165000 *+O   : :       *      *          O    *        *                 |
          |    O::      O                                   O               |
   160000 ++     : O             O                                        O |
          |      *   O      O O    O                    O     O      O O    O
          O      O        O          O        O  O O      O                 |
   155000 ++                            O O                                 |
          |                                          O                      |
   150000 ++----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                99349.14       +13.0%    112283.37  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                94794.47       +12.9%    107042.87  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
               169200.40        -6.0%    159010.97  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
               363344.00        +4.1%    378337.20  TOTAL lock_stat.&(&zone->lock)->rlock.contentions.__free_pages_ok


                     lock_stat.&(&n->list_lock)->rlock.contentions

   8.5e+06 ++---------------------------------------------------------------+
           |                    .*.....                   ....*......       |
     8e+06 ++                ...       .           ...*...           *......*
           *..... O        ..           *......*...   O                     |
           |     .    ...*.                    O              O      O      O
   7.5e+06 O+     *...                  O                                   |
           |                                                                |
     7e+06 ++                                                               |
           |                                                                |
   6.5e+06 ++                                                               |
           |                                                                |
           |                                                                |
     6e+06 ++                                                               |
           |                                                                |
   5.5e+06 ++--------------------O------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
              7973899.10        -6.7%   7438985.30  lkp-snb01/micro/hackbench/1600%-threads-socket
              7973899.10        -6.7%   7438985.30  TOTAL lock_stat.&(&n->list_lock)->rlock.contentions


             lock_stat.&(&n->list_lock)->rlock.contentions.get_partial_node

   2.16e+07 ++--------------------------------------------------------------+
   2.15e+07 ++                                                *.            |
            |                                                :  .           |
   2.14e+07 ++                                              :    ..         |
   2.13e+07 ++                                              :      .        |
            |                                              :        .O      |
   2.12e+07 ++                              ....*..       :                 |
   2.11e+07 *+.....                    .*...       ..    :           *......*
    2.1e+07 ++     *......*......  ....              ..  :                  |
            O                    *.                     :                   |
   2.09e+07 ++                                         *                    |
   2.08e+07 ++     O                                                        O
            |                    O                     O      O             |
   2.07e+07 ++                          O       O                           |
   2.06e+07 ++--------------------------------------------------------------+



             lock_stat.&(&n->list_lock)->rlock.contentions.get_partial_node

    1.4e+07 ++--------------------------------------------------------------+
   1.35e+07 ++                  .*......                   ...*......    ...*
            *.....          ....        *.......    ...*...          *...   |
    1.3e+07 ++    .O   ...*.                    *...   O                    |
   1.25e+07 O+     *...   O             O       O             O      O      O
            |                                                               |
    1.2e+07 ++                                                              |
   1.15e+07 ++                                                              |
    1.1e+07 ++                                                              |
            |                                                               |
   1.05e+07 ++                                                              |
      1e+07 ++                                                              |
            |                                                               |
    9.5e+06 ++                   O                                          |
      9e+06 ++--------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
             21093650.20        -1.2%  20846406.40  lkp-snb01/micro/hackbench/1600%-process-socket
             13228413.40        -6.5%  12371893.40  lkp-snb01/micro/hackbench/1600%-threads-socket
             34322063.60        -3.2%  33218299.80  TOTAL lock_stat.&(&n->list_lock)->rlock.contentions.get_partial_node


                                  vmstat.system.in

   1200 ++------------------------------------------------------------------+
   1180 ++                                                             O    |
        | O                  O                                            O |
   1160 ++                                                                  |
   1140 ++     O                      O    O                                |
        |                                                                   |
   1120 O+                               O                                  |
   1100 ++            O                         O             O             |
   1080 ++       O                  O             O                         |
        |           O   O                     O          O                  |
   1060 ++   O                    O                  O      O               |
   1040 ++     *.*.. .*.   O.*.*..*.*.*..      .*.  .*. .*..*.  O  O O     .*
        *.*.. +     *   *..*              .*..*   *.   *      *.*..*.*.*..* |
   1020 ++   *                           *                                  |
   1000 ++------------------------------------------------------------------+



                                  vmstat.system.in

   2400 ++------------------------------------------------------------------+
        |                                                                   |
   2200 ++                                                    O             |
        |           O                                                       |
        |               O             O                         O           |
   2000 ++     O                                                            |
        |                      O  O           O                             |
   1800 ++                               O             O O  O               |
        O                                  O      O  O             O        *
   1600 ++                 *   *    O                                      :|
        | O  O        O   + :  ::                                          :|
        *        O       +  :O: :               *        *..         O O .* |
   1400 ++             .*    ::  :       *.    + :      :      .*..   .*.   |
        | *..*.*.  .*.*      *   : .*. ..  *..*  :      :   *.*    *.*      |
   1200 ++-------*----------------*---*-----------*--*-*--------------------+



                                  vmstat.system.in

   1300 ++------------------------------------------------------------------+
        |                                                                   |
   1250 ++                  O                                               |
        |                                                                   |
        |                                                                   |
   1200 ++                                                   O      O       |
        |                             O         O                      O    |
   1150 O+ O           O            O    O                                  |
        |    O    O  O                        O                             |
   1100 ++                O                          O                      |
        |                                          O      O                 |
        |        .*..            O                  .*..          O      O  |
   1050 *+. .*..*    *.*..*.  .*.*..*.  .*.*..    .*    O   .*.*..*.*..     |
        |  *                *.        *.      *.*.      *.*.   O       *.*  O
   1000 ++------------------------------------------------------------------+



                                  vmstat.system.in

   1300 ++------------------------------------------------------------------+
        |           *                                                       |
        |           :                                                       |
   1250 ++          :                                                       |
        |          : :                                                      |
        |          : :                                                      |
   1200 ++         : :                                                      |
        |          : :                                                      |
   1150 ++        :  :                                                      |
        |         :  :                                                      |
        |         :   :                                                     |
   1100 ++        :   :                                                     |
        |    O O :    O        O    O           O O                       O |
        O.*..*.*.O  O *.O  O O   .*.  O  O O  O      O.*.O .*.*.O  O O O .*.O
   1050 *+-------*------*--*-*-*----*-*--*-*--*-*-*--*---*------*--*-*-*----*



                                  vmstat.system.in

   1220 ++------------------------------------------------------------------+
        | O                                                                 |
   1200 ++   O                           O      O           O               |
        |               O  O O    O        O           O                    |
   1180 ++          O          O                  O  O   O                  |
        O        O                  O O       O               O O  O O O    O
   1160 ++     O                                                            |
        |             O                                                   O |
   1140 ++                                                                  |
        |                                                                   |
   1120 ++                                                                  |
        | *..          .*..               .*..                 .*..*.       |
   1100 ++   *.*.*.. .*    *. .*..   .*..*    *. .*..*.*.*..*.*      *.  .* |
        *           *        *     .*           *                      *.  +|
   1080 ++------------------------*-----------------------------------------*



                                  vmstat.system.in

   1640 ++------------------------------------------------------------------+
        |                                                                   |
   1620 ++                 O O           O                                  |
        |           O                 O       O          O  O        O      O
        |      O O    O                              O O        O      O  O |
   1600 O+                     O  O             O O           O             |
        |    O          O                                          O        |
   1580 ++O                         O      O                                |
        |                                                                   |
   1560 ++                                                                  |
        |    *.    .*      *. .*..                                          |
        |  ..  *.*.  :   ..  *                   .*..          .*..  *.     |
   1540 ++*          : .*         *.*.*..*.*.. .*    *.*    *.*     +  *..*.|
        *             *                       *         + ..       *        *
   1520 ++-----------------------------------------------*------------------+



                                  vmstat.system.in

   1550 ++------------------------------------------------------------------+
        |                                O           O                      |
   1540 O+       O      O                     O                             |
   1530 ++          O      O      O               O                         |
        |                           O      O    O      O O         O O O    |
   1520 ++   O               O O                            O O           O O
   1510 ++O                                                     O           |
        |             O                                                     |
   1500 ++     O                      O                            *        |
   1490 ++                .*                                 .*. ..:        |
        |   .*.*        *.  :      .*.           .*..      .*   *   :       |
   1480 *+*.    +      +    : .*..*   *..*.*..*.*    *   *.         :  *    *
   1470 ++       *.. .*      *                        + +            :+ +  :|
        |           *                                  *             *   + :|
   1460 ++----------------------------------------------------------------*-+



                                  vmstat.system.in

   5500 ++------------------------------------------------------------------+
        |                      O           O  O             O          O    |
   5000 ++     O                                                            |
        |        O           O                           O    O           O |
   4500 ++          O                                O             O        O
        O O           O O  O      O             O      O        O           |
   4000 ++   O                      O O                              O      |
        |                                O        O                         |
   3500 ++                                                                  |
        |                                                                   |
   3000 ++                                                                  |
        |                                  *..           *..                |
   2500 *+    .*.  .*       .*    *.      +    .*.      +   *.         *..*.|
        | *..*   *.  + .*..*  + ..  *.*..*    *   *..*.*      *.*..*. +     *
   2000 ++------------*--------*-------------------------------------*------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 1033.25        +5.6%      1091.39  kbuildx/micro/netperf/120s-200%-TCP_RR
                 1339.95       +30.4%      1746.92  kbuildx/micro/netperf/120s-200%-TCP_STREAM
                 1036.18        +7.8%      1117.29  kbuildx/micro/netperf/120s-200%-UDP_RR
                 1064.41        +0.9%      1073.76  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                 1097.78        +7.5%      1180.30  lkp-a04/micro/netperf/120s-200%-TCP_RR
                 1538.97        +4.2%      1603.88  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 1479.54        +2.9%      1523.00  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
                 2348.58       +90.7%      4477.99  vpx/micro/hackbench/1600%-threads-socket
                10938.67       +26.3%     13814.52  TOTAL vmstat.system.in


                                  vmstat.system.cs

   6000 ++------------------------------------------------------------------+
        |      *                      *    *..           *                  |
   5800 ++*.. + :   *.*.*..   .*..*  : +  :    .*.  .*. + +   *.*..*.       |
        |+   *  : ..       *.*     + :  + :   *   *.   *   + +       *.*..* |
   5600 *+       *                  *    *                  *               |
        |                                                                   |
   5400 ++                                                                  |
        |                                                                   |
   5200 ++                                                                  |
        |                                                   O               |
   5000 ++              O                                O    O             |
        O        O    O        O         O O    O O                    O  O |
   4800 ++O  O      O        O    O   O       O        O             O      O
        |      O           O        O                O          O  O        |
   4600 ++------------------------------------------------------------------+



                                  vmstat.system.cs

   9800 ++------------------------------------------------------------------+
        |            .*.                                                    |
   9600 ++*..   .*..*   *..           *..*.*..*.    .*.*.*..    *..*. .*.. .*
        *    *.*           *.*.*..*. +          *.*.        *. +     *    * |
   9400 ++                          *                         *             |
        |                                                                   |
   9200 ++                                                                  |
        |                                                                   |
   9000 ++                                                                  |
        |                                                                   |
   8800 ++                                                                  |
        O O  O O        O           O         O   O  O      O O    O O      O
   8600 ++       O    O    O O O           O    O      O O             O    |
        |           O             O   O  O                      O         O |
   8400 ++------------------------------------------------------------------+



                                  vmstat.system.cs

   6400 ++------------------------------------------------------------------+
        |                             *                                     |
   6200 ++*..         *.    .*.  .*. + +       .*                           |
   6000 ++      .*.. +  *..*   *.   *   +    .*  +  .*.*       .*..*   *..  |
        *    *.*    *                    *.*.     *.    +  .*.*     + +     |
   5800 ++                                               *.          *    *.|
   5600 ++                                                                  *
        |                                                                   |
   5400 ++                                                                  |
   5200 ++                                 O           O                    |
        O    O        O    O      O           O                        O    |
   5000 ++     O O  O   O             O              O        O O    O    O |
   4800 ++O                  O O    O    O      O        O  O      O        O
        |                                         O                         |
   4600 ++------------------------------------------------------------------+



                                  vmstat.system.cs

   7150 ++------------------------------------------------------------------+
   7100 ++*      *                                          *       .*      |
        |+ +     :+                     .*            .*   : + .*..*  :     |
   7050 *+  +   :  +              *. .*.  +      .*..*  :  :  * O     :     *
   7000 ++   *. :   *     .*    ..  *      *.. .*       : :            *.. +|
   6950 ++O    *     + .*.  + .*              *          *  O      O      * |
   6900 ++            *    O * O                                       O  O |
        O        O  O        O                O O      O      O             O
   6850 ++   O        O O         O                  O   O           O      |
   6800 ++                          O    O        O                         |
   6750 ++                                                                  |
   6700 ++                                                                  |
        |                                                                   |
   6650 ++     O                      O                                     |
   6600 ++------------------------------------------------------------------+



                                    vmstat.system.cs

   2.43e+06 ++--------------------------------------------------------------+
            |                                  .*                           *
   2.42e+06 ++                              ...  :                        ..|
   2.41e+06 ++            *..             ..      :                     ..  |
            |            :   .          *.         :                  ..    |
    2.4e+06 ++          :     .       ..        O   :               .*      |
            |           :      ..   ..              :             ..        |
   2.39e+06 *+.        :          ..                 :          ..          |
            |  .      :          *                    :    ...*.            |
   2.38e+06 ++  .  O :                                 *...          O      |
   2.37e+06 ++   ..  :                  O                                   |
            |       :                                         O             |
   2.36e+06 ++     *                                   O                    O
            O                    O                                          |
   2.35e+06 ++------------O-------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                 5747.60       -15.7%      4847.23  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
                 9551.08        -9.7%      8620.69  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
                 5960.08       -16.3%      4985.65  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
                 7014.34        -2.1%      6865.74  lkp-a04/micro/netperf/120s-200%-UDP_STREAM
              2396987.08        -1.2%   2367695.46  lkp-snb01/micro/hackbench/1600%-threads-pipe
              2425260.17        -1.3%   2393014.77  TOTAL vmstat.system.cs


                                  time.user_time

   600 ++-------------------------------------------------------------------+
       *.......*......*.......*.......*......*.......*.....             ....*
   595 ++                                                  ..    ...*...    |
   590 ++                                                    *...           |
       |                                                                    |
   585 ++                                                                   |
   580 ++                                                                   |
       |                                                                    |
   575 ++                                                                   |
   570 ++                     O       O                                     |
       O                                                     O              |
   565 ++             O                              O              O       |
   560 ++      O                             O                              |
       |                                                                    |
   555 ++-------------------------------------------------------------------O



                                  time.user_time

   575 ++-------------------------------------------------------------------+
       *.......                                                             |
   570 ++      *......                                                    ..*
   565 ++             *.......*.......*......                         ....  |
       |                                     *.......*.......*......*.      |
   560 ++                                                                   |
       |                                                                    |
   555 ++                                                                   |
       |                                                                    |
   550 ++                                                                   |
   545 ++                     O                                             O
       O       O                      O                      O              |
   540 ++             O                      O       O              O       |
       |                                                                    |
   535 ++-------------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  595.79        -5.1%       565.31  lkp-snb01/micro/hackbench/1600%-process-socket
                  565.98        -4.0%       543.24  lkp-snb01/micro/hackbench/1600%-threads-pipe
                 1161.77        -4.6%      1108.55  TOTAL time.user_time


                                   iostat.cpu.user

    2.2 ++------------------------------------------------------------------+
   2.18 ++                                ....*......                       |
        *.......                   ...*...           *.......    ...*.......|
   2.16 ++      *......*.......*...                          *...           *
   2.14 ++                                                                  |
        |                                                                   |
   2.12 ++                                                                  |
    2.1 ++                                                                  |
   2.08 ++                                                                  |
        |                      O      O                                     |
   2.06 ++             O                                     O      O       |
   2.04 O+                                                                  |
        |       O                             O      O                      |
   2.02 ++                                                                  O
      2 ++------------------------------------------------------------------+



                                   iostat.cpu.user

   2.92 ++------------------------------------------------------------------+
        |                      *.....                                       |
    2.9 ++  ....*..          ..      .                                      |
   2.88 *+..       ..      ..         *...                 ..*...           |
        |            ..   .               ...          ....      ..       ..*
   2.86 ++              ..                   .    ...*.            .  ....  |
   2.84 ++             *                      *...                  *.      |
        |                                                                   |
   2.82 ++                                                                  O
    2.8 ++                                                                  |
        O                      O                                            |
   2.78 ++      O      O              O                      O              |
   2.76 ++                                    O      O                      |
        |                                                           O       |
   2.74 ++------------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                    2.17        -5.5%         2.05  lkp-snb01/micro/hackbench/1600%-process-socket
                    2.87        -3.1%         2.78  lkp-snb01/micro/hackbench/1600%-threads-pipe
                    5.04        -4.1%         4.83  TOTAL iostat.cpu.user


                             time.voluntary_context_switches

    8.2e+08 ++--------------------------------------------------------------+
            |             *                                                 |
   8.15e+08 ++           : +                ....*.                         .*
    8.1e+08 ++          :   +           *...      .                    .... |
            |           :    +        ..           ..               .*.     |
   8.05e+08 ++         :      +      .               .          ....        |
            *..       :        +    .                 .      .*.            |
      8e+08 ++ ..    :          + ..                     ....               |
            |    ..  :           *                     *.                   |
   7.95e+08 ++      :                                         O      O      |
    7.9e+08 ++     *                    O                                   |
            O                                                               O
   7.85e+08 ++            O      O                                          |
            |      O                                   O                    |
    7.8e+08 ++----------------------------------O---------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
            805435366.50        -2.2% 787433678.70  lkp-snb01/micro/hackbench/1600%-threads-pipe
            805435366.50        -2.2% 787433678.70  TOTAL time.voluntary_context_switches


                            time.involuntary_context_switches

    6.7e+08 ++--------------------------------------------------------------+
            |                                                               |
   6.65e+08 ++            *..                 ..*..                         |
            |            :   ..           ....     ..                    ...*
            |           :      ..      .*.           ..             .*...   |
    6.6e+08 ++         :           ....                         ....        |
            |          :         *.                    *......*.            |
   6.55e+08 *+..      :                                                     |
            |   ..   :                                                      |
    6.5e+08 ++    . :                                                       |
            |      *                                          O             |
            |                           O                            O      |
   6.45e+08 O+     O      O      O                     O                    O
            |                                                               |
    6.4e+08 ++----------------------------------O---------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
            659247983.60        -2.2% 644839552.10  lkp-snb01/micro/hackbench/1600%-threads-pipe
            659247983.60        -2.2% 644839552.10  TOTAL time.involuntary_context_switches


                                   vmstat.cpu.us

   6.2 ++-------------------------------------------------------------------+
       |                                                                    |
   6.1 ++     *                                   *                         |
       |      ::                           *.    +:           *        O    |
       *     :  :                         :  *. +  :   *     : :            *
     6 ++    :  : .*.                     :    *   :  + :    : :            |
       |:    :   *   *    *              :          :+  :   :   :     .*   :|
   5.9 ++   :        O+  : +        O *.*    O      *    *. :   *.*..*  :  :|
       | O .*          + :O *..*.*.. :O                    *             : :O
   5.8 ++*.             *   O        :  O           O  O O      O    O   :O |
       |                O           *                      O  O           * |
       |      O    O                           O  O                         |
   5.7 O+                                  O                                |
       |                       O                                            |
   5.6 ++---------------------------------------------------------O---------+



                                    vmstat.cpu.us

   2.06 ++------------------------------------------------------------------+
        |                      *...                                         |
   2.04 *+.                  ..    ..                                       |
        |  ..              ..        .            ...*...                   |
   2.02 ++   ..           .           *.......*...       ...                |
        |      .        ..                                  .    ...*.......*
      2 ++      *......*                                     *...           |
        |                                                                   |
   1.98 ++                                                                  |
        |                                                                   |
   1.96 ++                                                          O       |
        |              O       O                                            |
   1.94 O+                            O                                     |
        |       O                             O      O       O              |
   1.92 ++------------------------------------------------------------------O



                                    vmstat.cpu.us

   2.94 ++------------------------------------------------------------------+
        |                                                                   |
   2.93 ++     .*..                                                     ....*
        |   ...    ..         .*......                             .*...    |
   2.92 ++..         ..    ...        *..                        ..         |
   2.91 *+               ..              ..                    ..           |
        |              *.                  ..            ....*.             |
    2.9 ++                                   .    ...*...                   O
        |                                     *...                          |
   2.89 ++                                                                  |
        O                             O       O                             |
   2.88 ++      O              O                                            |
   2.87 ++                                                   O      O       |
        |              O                                                    |
   2.86 ++-------------------------------------------O----------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                    5.93        -2.2%         5.81  lkp-a04/micro/netperf/120s-200%-TCP_CRR
                    2.02        -4.0%         1.94  lkp-snb01/micro/hackbench/1600%-process-socket
                    2.91        -1.3%         2.88  lkp-snb01/micro/hackbench/1600%-threads-pipe
                   10.87        -2.3%        10.62  TOTAL vmstat.cpu.us


                                 time.elapsed_time

   616 *+-------------------------------------------------------------------+
       |+             *.                                                    |
   614 +++           +  ..                                                  |
       |  +         +     .                                                 |
   612 ++  +       +       .                                                |
       |    +     +         ..      ..*......*.......            ...*.......|
   610 ++    +   +              ....                 *.......*...           *
       |      + +             *.                                    O       |
   608 ++      *                                             O              |
       O              O       O                                             |
   606 ++                             O                                     O
       |       O                                     O                      |
   604 ++                                                                   |
       |                                     O                              |
   602 ++-------------------------------------------------------------------+



                                 time.elapsed_time

   660 ++------------------------O------------------------------------------+
       |                            O             O      O                  O
   650 O+                   O              O   O              O O           |
       |    O O    O O  *.O                                               O |
       |                : *.*  O   .*.       *             *..       *     .*
   640 ++O       O.*   :O   :    *.   * O   +O    *    O  :   *    ..O    * |
       |         * :   :     :   :     :   *  :  :: O     :   :   *   :  :  |
   630 ++*      :   : :      :  :      : ..   :  : :     *     :  :   :  :  |
       | ::     :   : :       : :       *     : :  :    :      : :    : :   |
   620 ++ :    :     :        : :              ::  :    :      : :     ::   |
       |:  :  *      *         :               *    : .*        :      *    |
       *   : +                 *                    *.          *           |
   610 ++   *                                                               |
       |                                                                    |
   600 ++--------------------------------------------------O-----------O----+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                  611.08        -0.8%       606.20  lkp-snb01/micro/hackbench/1600%-threads-pipe
                  630.55        +2.0%       643.10  vpx/micro/hackbench/1600%-process-pipe
                 1241.63        +0.6%      1249.30  TOTAL time.elapsed_time


                                  time.system_time

   18850 ++-----------------------------------------------------------------+
         |                                                                  |
   18800 *+             *.                                                  |
   18750 ++            .  .                                                 |
         | +          .    ..                                               |
   18700 ++ +       ..       .                                              |
   18650 ++  +     .          .      ..*......*.......           ....*......|
         |    +   .              ....                 *......*...    O      *
   18600 ++    + .             *.                            O              |
   18550 ++     *       O                                                   |
         O                     O       O                                    O
   18500 ++                                           O                     |
   18450 ++     O                                                           |
         |                                    O                             |
   18400 ++-----------------------------------------------------------------+

    62470419e993f8d9d93d      7d9ffa8961482232d964  
------------------------  ------------------------  
                18656.34        -0.7%     18527.16  lkp-snb01/micro/hackbench/1600%-threads-pipe
                18656.34        -0.7%     18527.16  TOTAL time.system_time
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ