linux-kernel - LLC cache miss increases with the increases of hugepages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Mon, 6 Mar 2017 10:53:06 +0800
From:   linhaifeng <haifeng.lin@...wei.com>
To:     <nyc@...omorphy.com>, <tglx@...utronix.de>, <mingo@...hat.com>,
        <hpa@...or.com>
CC:     <linux-kernel@...r.kernel.org>
Subject: LLC cache miss increases with the increases of hugepages

Hi，

I did a network forward test with ovs-dpdk, and found that LLC cache miss
increases with the increases of hugepages. ovs-dpdk only use 4 hugepages,
hugepage size is 1G.Why LLC cache miss increases ? I just echo more hugepages
but haven't been used yet.

some informations:

CPU: Intel(R) Xeon(R) CPU E5-2620 v3@ 2.40GHz
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              15360K

echo 10 > /proc/sys/vm/nr_hugepages
ovs-dpdk's LLC-load-misses:3.88%
ovs-dpdk's iTLB-load-misses: 4.37%

echo 100 > /proc/sys/vm/nr_hugepages
reboot VM restart test.
ovs-dpdk's LLC-load-misses:31.55%
ovs-dpdk's iTLB-load-misses: 36.36%

I use perf to watch LLC-loads when not run test:

Samples: 1M of event 'LLC-loads', Event count (approx.): 57240206
Overhead  Shared Object                 Symbol
   9.19%  [kernel]                      [k] copy_page_rep
   5.03%  [kernel]                      [k] ktime_get
   4.06%  [unknown]                     [.] 0x000000000043e39d
   2.65%  bash                          [.] 0x000000000003e39d
   2.13%  [kernel]                      [k] rcu_check_callbacks

The function copy_page_rep seems like would prefetch all hugepages?