lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:	Sat, 1 Feb 2014 20:00:03 +0800
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	Dave Hansen <dave@...1.net>
Cc:	Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, cl@...ux-foundation.org,
	penberg@...nel.org, mpm@...enic.com, pshelar@...ira.com
Subject: Re: [PATCH] mm: slub: fix page->_count corruption (again)

On Tue, Jan 28, 2014 at 03:52:47PM -0800, Dave Hansen wrote:
> On 01/28/2014 03:29 PM, Andrew Morton wrote:
> > On Tue, 28 Jan 2014 15:17:22 -0800 Dave Hansen <dave@...1.net> wrote:
> > This code is borderline insane.
> 
> No argument here.
> 
> > Yes, struct page is special and it's worth spending time and doing
> > weird things to optimise it.  But sheesh.
> > 
> > An alternative is to make that cmpxchg quietly go away.  Is it more
> > trouble than it is worth?
> 
> It has measurable performance benefits, and the benefits go up as the
> cost of en/disabling interrupts goes up (like if it takes you a hypercall).
> 
> Fengguang, could you run a set of tests for the top patch in this branch
> to see if we'd be giving much up by axing the code?
> 
> 	https://github.com/hansendc/linux/tree/slub-nocmpxchg-for-Fengguang-20140128
> 
> I was talking with one of the distros about turning it off as well.
> They mentioned that they saw a few performance regressions when it was
> turned off.  I'll share details when I get them.

Hi Dave,

Here are the test results, which look pretty good:

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
    208389 ~ 3%      +5.2%     219282 ~ 1%  lkp-snb01/micro/hackbench/1600%-process-pipe
    155284 ~ 1%      +1.9%     158203 ~ 0%  lkp-snb01/micro/hackbench/1600%-threads-pipe
    363673 ~ 2%      +3.8%     377485 ~ 0%  TOTAL hackbench.throughput

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       174 ~ 0%      +2.3%        178 ~ 0%  nhm-white/sysbench/oltp/600s-100%-1000000
       174 ~ 0%      +2.3%        178 ~ 0%  TOTAL oltp.request_latency_max_ms

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      2077 ~ 0%      +2.1%       2122 ~ 0%  lkp-a04/micro/netperf/120s-200%-TCP_RR
      2553 ~ 0%      +1.2%       2584 ~ 0%  lkp-a04/micro/netperf/120s-200%-UDP_RR
      4630 ~ 0%      +1.6%       4706 ~ 0%  TOTAL netperf.Throughput_tps

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      8181 ~ 0%      +0.8%       8247 ~ 0%  nhm8/micro/dbench/100%
      8181 ~ 0%      +0.8%       8247 ~ 0%  TOTAL dbench.throughput-MB/sec

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
  37597266 ~ 0%      +0.4%   37736192 ~ 0%  nhm-white/sysbench/oltp/600s-100%-1000000
  37597266 ~ 0%      +0.4%   37736192 ~ 0%  TOTAL oltp.rw_requets

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
   2685519 ~ 0%      +0.4%    2695442 ~ 0%  nhm-white/sysbench/oltp/600s-100%-1000000
   2685519 ~ 0%      +0.4%    2695442 ~ 0%  TOTAL oltp.transactions

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     28567 ~33%     -67.4%       9319 ~38%  lkp-a04/micro/netperf/120s-200%-TCP_CRR
     28567 ~33%     -67.4%       9319 ~38%  TOTAL cpuidle.C2.time

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
    134306 ~ 3%     +22.8%     164973 ~ 4%  xps2/micro/pigz/100%
    134306 ~ 3%     +22.8%     164973 ~ 4%  TOTAL cpuidle.C1-NHM.time

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      8702 ~29%     -40.8%       5148 ~47%  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
     27110 ~43%     -64.4%       9660 ~41%  lkp-a04/micro/netperf/120s-200%-UDP_RR
     35812 ~40%     -58.7%      14808 ~43%  TOTAL cpuidle.C3.time

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
        33 ~20%     -42.4%         19 ~19%  lkp-a04/micro/netperf/120s-200%-TCP_CRR
        33 ~20%     -42.4%         19 ~19%  TOTAL cpuidle.C2.usage

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     48500 ~14%     +39.8%      67818 ~10%  lkp-snb01/micro/hackbench/1600%-threads-socket
     48500 ~14%     +39.8%      67818 ~10%  TOTAL numa-meminfo.node1.Active(file)

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     12125 ~14%     +39.7%      16937 ~10%  lkp-snb01/micro/hackbench/1600%-threads-socket
     12125 ~14%     +39.7%      16937 ~10%  TOTAL numa-vmstat.node1.nr_active_file

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     13387 ~19%     -40.7%       7933 ~ 3%  xps2/micro/pigz/100%
     13387 ~19%     -40.7%       7933 ~ 3%  TOTAL interrupts.45:PCI-MSI-edge.eth0

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     21050 ~16%     +27.4%      26818 ~ 4%  lkp-snb01/micro/hackbench/1600%-threads-socket
     21050 ~16%     +27.4%      26818 ~ 4%  TOTAL numa-meminfo.node1.SReclaimable

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      5271 ~17%     +26.9%       6687 ~ 4%  lkp-snb01/micro/hackbench/1600%-threads-socket
      5271 ~17%     +26.9%       6687 ~ 4%  TOTAL numa-vmstat.node1.nr_slab_reclaimable

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
    161543 ~ 8%     -18.7%     131356 ~ 7%  nhm-white/sysbench/oltp/600s-100%-1000000
    161543 ~ 8%     -18.7%     131356 ~ 7%  TOTAL cpuidle.C1E-NHM.usage

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      1039 ~ 1%     -17.7%        856 ~13%  lkp-a04/micro/netperf/120s-200%-TCP_RR
      1039 ~ 1%     -17.7%        856 ~13%  TOTAL slabinfo.proc_inode_cache.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
   8762198 ~ 7%     -19.3%    7075254 ~ 6%  nhm-white/sysbench/oltp/600s-100%-1000000
   8762198 ~ 7%     -19.3%    7075254 ~ 6%  TOTAL cpuidle.C1E-NHM.time

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
    285124 ~12%     -22.3%     221550 ~ 4%  nhm-white/sysbench/oltp/600s-100%-1000000
    285124 ~12%     -22.3%     221550 ~ 4%  TOTAL interrupts.RES

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
        25 ~ 9%     +43.4%         36 ~11%  lkp-snb01/micro/hackbench/1600%-threads-pipe
        25 ~ 9%     +43.4%         36 ~11%  TOTAL cpuidle.C6-SNB.usage

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      1337 ~ 9%     +14.9%       1536 ~ 9%  lkp-a04/micro/netperf/120s-200%-TCP_CRR
      2932 ~ 6%     -26.6%       2153 ~ 3%  nhm-white/sysbench/oltp/600s-100%-1000000
      4269 ~ 7%     -13.6%       3690 ~ 5%  TOTAL slabinfo.kmalloc-128.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      2932 ~ 6%     -19.0%       2374 ~ 1%  nhm-white/sysbench/oltp/600s-100%-1000000
      2932 ~ 6%     -19.0%       2374 ~ 1%  TOTAL slabinfo.kmalloc-128.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       729 ~ 8%     -20.8%        577 ~ 5%  nhm-white/sysbench/oltp/600s-100%-1000000
       729 ~ 8%     -20.8%        577 ~ 5%  TOTAL slabinfo.signal_cache.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      1042 ~ 0%     -15.2%        884 ~11%  lkp-a04/micro/netperf/120s-200%-TCP_RR
      1042 ~ 0%     -15.2%        884 ~11%  TOTAL slabinfo.proc_inode_cache.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      2625 ~ 3%     +21.9%       3202 ~ 1%  lkp-a04/micro/netperf/120s-200%-TCP_CRR
      3257 ~11%     -12.8%       2839 ~10%  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
      3535 ~ 3%     +12.4%       3973 ~ 5%  nhm-white/sysbench/oltp/600s-100%-1000000
      3775 ~ 3%     +10.7%       4177 ~ 6%  nhm8/micro/dbench/100%
     13193 ~ 5%      +7.6%      14192 ~ 5%  TOTAL slabinfo.anon_vma.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      2625 ~ 3%     +21.9%       3202 ~ 1%  lkp-a04/micro/netperf/120s-200%-TCP_CRR
      3257 ~11%     -12.8%       2839 ~10%  lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
      3535 ~ 3%     +12.4%       3973 ~ 5%  nhm-white/sysbench/oltp/600s-100%-1000000
      3775 ~ 3%     +10.7%       4177 ~ 6%  nhm8/micro/dbench/100%
     13193 ~ 5%      +7.6%      14192 ~ 5%  TOTAL slabinfo.anon_vma.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       456 ~ 4%     -12.8%        398 ~ 5%  xps2/micro/pigz/100%
       456 ~ 4%     -12.8%        398 ~ 5%  TOTAL cpuidle.C1-NHM.usage

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      1309 ~ 5%     +12.1%       1468 ~ 3%  lkp-snb01/micro/hackbench/1600%-threads-socket
      1309 ~ 5%     +12.1%       1468 ~ 3%  TOTAL numa-vmstat.node0.nr_alloc_batch

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       332 ~ 5%     +20.3%        399 ~ 6%  lkp-snb01/micro/hackbench/1600%-threads-pipe
       332 ~ 5%     +20.3%        399 ~ 6%  TOTAL numa-vmstat.node1.nr_page_table_pages

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      1340 ~ 5%     +19.2%       1597 ~ 6%  lkp-snb01/micro/hackbench/1600%-threads-pipe
      1340 ~ 5%     +19.2%       1597 ~ 6%  TOTAL numa-meminfo.node1.PageTables

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       570 ~ 0%     +11.0%        633 ~ 4%  nhm8/micro/dbench/100%
       570 ~ 0%     +11.0%        633 ~ 4%  TOTAL cpuidle.C6-NHM.usage

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     21405 ~ 2%     +12.6%      24105 ~ 1%  lkp-snb01/micro/hackbench/1600%-threads-pipe
     21405 ~ 2%     +12.6%      24105 ~ 1%  TOTAL slabinfo.kmalloc-192.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     21567 ~ 2%     +12.5%      24256 ~ 1%  lkp-snb01/micro/hackbench/1600%-threads-pipe
     21567 ~ 2%     +12.5%      24256 ~ 1%  TOTAL slabinfo.kmalloc-192.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     53135 ~ 1%      -9.6%      48060 ~ 3%  lkp-snb01/micro/hackbench/1600%-threads-pipe
     53135 ~ 1%      -9.6%      48060 ~ 3%  TOTAL numa-vmstat.node0.nr_active_anon

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       952 ~ 5%     -14.3%        816 ~ 0%  lkp-a04/micro/netperf/120s-200%-UDP_RR
       952 ~ 5%     -14.3%        816 ~ 0%  TOTAL slabinfo.Acpi-Namespace.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       952 ~ 5%     -14.3%        816 ~ 0%  lkp-a04/micro/netperf/120s-200%-UDP_RR
       952 ~ 5%     -14.3%        816 ~ 0%  TOTAL slabinfo.Acpi-Namespace.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       949 ~ 3%     +12.3%       1066 ~ 3%  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
       949 ~ 3%     +12.3%       1066 ~ 3%  TOTAL slabinfo.buffer_head.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       949 ~ 3%     +12.3%       1066 ~ 3%  lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
       949 ~ 3%     +12.3%       1066 ~ 3%  TOTAL slabinfo.buffer_head.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
    211422 ~ 1%      -8.9%     192562 ~ 4%  lkp-snb01/micro/hackbench/1600%-threads-pipe
    211422 ~ 1%      -8.9%     192562 ~ 4%  TOTAL numa-meminfo.node0.Active(anon)

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      8874 ~ 2%      +7.7%       9557 ~ 2%  nhm8/micro/dbench/100%
      8874 ~ 2%      +7.7%       9557 ~ 2%  TOTAL slabinfo.kmalloc-8.num_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      8874 ~ 2%      +7.7%       9557 ~ 2%  nhm8/micro/dbench/100%
      8874 ~ 2%      +7.7%       9557 ~ 2%  TOTAL slabinfo.kmalloc-8.active_objs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       196 ~15%     -17.7%        161 ~ 2%  nhm-white/sysbench/oltp/600s-100%-1000000
       196 ~15%     -17.7%        161 ~ 2%  TOTAL cpuidle.POLL.usage

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     15.08 ~ 4%      +5.1%      15.85 ~ 5%  lkp-snb01/micro/hackbench/1600%-threads-pipe
      7.38 ~ 0%      +1.7%       7.51 ~ 0%  nhm-white/sysbench/oltp/600s-100%-1000000
     22.46 ~ 3%      +4.0%      23.35 ~ 3%  TOTAL boottime.dhcp

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       690 ~ 3%      +4.7%        722 ~ 2%  lkp-snb01/micro/hackbench/1600%-threads-pipe
       648 ~ 3%      +6.7%        692 ~ 4%  lkp-snb01/micro/hackbench/1600%-threads-socket
      1339 ~ 3%      +5.6%       1414 ~ 3%  TOTAL boottime.idle

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
 9.102e+08 ~ 0%      -4.9%  8.655e+08 ~ 1%  lkp-snb01/micro/hackbench/1600%-process-pipe
    665924 ~ 2%      -4.4%     636628 ~ 3%  xps2/micro/pigz/100%
 9.109e+08 ~ 0%      -4.9%  8.661e+08 ~ 1%  TOTAL time.involuntary_context_switches

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     25.37 ~ 3%      +2.6%      26.02 ~ 2%  lkp-snb01/micro/hackbench/1600%-threads-pipe
     24.11 ~ 2%      +5.7%      25.49 ~ 3%  lkp-snb01/micro/hackbench/1600%-threads-socket
     49.48 ~ 2%      +4.1%      51.51 ~ 3%  TOTAL boottime.boot

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
    962287 ~ 6%      -6.3%     902061 ~ 2%  lkp-snb01/micro/hackbench/1600%-process-pipe
      3996 ~ 1%      -2.8%       3886 ~ 1%  xps2/micro/pigz/100%
    966283 ~ 6%      -6.2%     905948 ~ 2%  TOTAL vmstat.system.in

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
  46760735 ~ 4%      +5.2%   49172076 ~ 1%  lkp-snb01/micro/hackbench/1600%-process-pipe
  46760735 ~ 4%      +5.2%   49172076 ~ 1%  TOTAL time.minor_page_faults

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     33216 ~ 0%      +1.9%      33856 ~ 0%  lkp-a04/micro/netperf/120s-200%-TCP_RR
     21517 ~ 0%      -0.9%      21332 ~ 0%  lkp-a04/micro/netperf/120s-200%-TCP_STREAM
     40720 ~ 0%      +1.1%      41187 ~ 0%  lkp-a04/micro/netperf/120s-200%-UDP_RR
   4694243 ~ 1%      -5.4%    4440006 ~ 0%  lkp-snb01/micro/hackbench/1600%-process-pipe
   5944020 ~ 0%      -1.5%    5856248 ~ 0%  lkp-snb01/micro/hackbench/1600%-threads-pipe
    485705 ~ 2%      -5.5%     459043 ~ 3%  lkp-snb01/micro/hackbench/1600%-threads-socket
      8367 ~ 0%      -1.5%       8237 ~ 1%  xps2/micro/pigz/100%
  11227790 ~ 1%      -3.3%   10859911 ~ 0%  TOTAL vmstat.system.cs

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
 1.949e+09 ~ 2%      -4.9%  1.855e+09 ~ 0%  lkp-snb01/micro/hackbench/1600%-process-pipe
  2.61e+08 ~ 2%      -6.2%  2.448e+08 ~ 3%  lkp-snb01/micro/hackbench/1600%-threads-socket
  85908841 ~ 0%      +0.4%   86231490 ~ 0%  nhm-white/sysbench/oltp/600s-100%-1000000
   1804105 ~ 0%      -0.5%    1795191 ~ 0%  xps2/micro/pigz/100%
 2.298e+09 ~ 2%      -4.8%  2.187e+09 ~ 0%  TOTAL time.voluntary_context_switches

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      1396 ~ 0%      +2.6%       1432 ~ 1%  lkp-snb01/micro/hackbench/1600%-threads-pipe
      1356 ~ 0%      -2.1%       1328 ~ 1%  lkp-snb01/micro/hackbench/1600%-threads-socket
      2752 ~ 0%      +0.3%       2760 ~ 1%  TOTAL time.user_time

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
      2750 ~ 0%      -0.9%       2726 ~ 0%  lkp-snb01/micro/hackbench/1600%-process-pipe
      2750 ~ 0%      -0.9%       2726 ~ 0%  TOTAL time.percent_of_cpu_this_job_got

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       161 ~ 0%      +0.3%        162 ~ 0%  lkp-snb01/micro/hackbench/1600%-process-pipe
       166 ~ 0%      +0.3%        166 ~ 0%  lkp-snb01/micro/hackbench/1600%-threads-pipe
       327 ~ 0%      +0.3%        328 ~ 0%  TOTAL turbostat.Cor_W

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
     93.10 ~ 0%      -0.4%      92.75 ~ 0%  lkp-snb01/micro/hackbench/1600%-process-pipe
     93.10 ~ 0%      -0.4%      92.75 ~ 0%  TOTAL turbostat.%c0

d891ea23d5203e5  29a5e4f2583c06bcb0529a75b  
---------------  -------------------------  
       189 ~ 0%      +0.2%        189 ~ 0%  lkp-snb01/micro/hackbench/1600%-process-pipe
       194 ~ 0%      +0.3%        194 ~ 0%  lkp-snb01/micro/hackbench/1600%-threads-pipe
       383 ~ 0%      +0.3%        384 ~ 0%  TOTAL turbostat.Pkg_W

Thanks,
Fengguang

View attachment "x86_64-lkp" of type "text/plain" (80581 bytes)

Powered by blists - more mailing lists