linux-kernel - page faults affects the performance

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAEK8JBCZ9HOHs9N6mdjg-q5pr4KZe3kWnrYKgW-huo8Yg9FZmQ@mail.gmail.com>
Date:   Tue, 31 Jul 2018 15:54:08 +0800
From:   Feng Li <lifeng1519@...il.com>
To:     linux-kernel <linux-kernel@...r.kernel.org>
Subject: page faults affects the performance

Hi experts,


When I use fio to write our distribution storage, I find a lot of page
faults that affect the performance.

My OS is CentOS7,  anyone could give suggestions on how to root cause
this issue? The CPU is running at 100%.

[root@58 15:50:57 ~]$uname -a
Linux 58 3.10.0-693.11.6.el7.smartx.1.x86_64 #1 SMP Tue Jan 16
22:09:10 CST 2018 x86_64 x86_64 x86_64 GNU/Linux

Thanks in advance.

This is the perf report from "perf record -g -p 9881"

+   64.68%     0.01%  zbs-chunkd       [k] system_call_fastpath

       ▒
+   42.86%     0.01%  zbs-chunkd       [k] inet_recvmsg

       ▒
+   42.84%     0.00%  zbs-chunkd       [k] sys_recvfrom

       ▒
+   42.83%     0.13%  zbs-chunkd       [k] tcp_recvmsg

       ▒
+   42.83%     0.01%  zbs-chunkd       [k] SYSC_recvfrom

       ▒
+   42.75%     0.02%  zbs-chunkd       [k] sock_recvmsg

       ▒
+   39.52%     0.09%  zbs-chunkd       [k] skb_copy_datagram_iovec

       ▒
-   31.97%     7.18%  zbs-chunkd       [k] page_fault

       ▒
   - 24.79% page_fault

       ▒
      + 24.78% do_page_fault

       ▒
   - 7.17% __libc_recv

       ▒
      - 7.17% system_call_fastpath

       ▒
           sys_recvfrom

       ▒
           SYSC_recvfrom

       ▒
           sock_recvmsg

       ▒
           inet_recvmsg

       ▒
           tcp_recvmsg

       ▒
           skb_copy_datagram_iovec

       ▒
           page_fault

       ▒


+   24.79%     0.04%  zbs-chunkd       [k] do_page_fault


This is the output of "perf record -e page-faults -ag":

Samples: 132K of event 'page-faults', Event count (approx.): 9249205
  Children      Self  Command          Shared Object                 Symbol
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] skb_copy_datagram_iovec
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] tcp_recvmsg
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] inet_recvmsg
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] sock_recvmsg
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] SYSC_recvfrom
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] sys_recvfrom
+   78.92%     0.00%  zbs-chunkd       [kernel.kallsyms]
[k] system_call_fastpath
+   78.92%    78.92%  zbs-chunkd       [kernel.kallsyms]
[k] copy_user_enhanced_fast_string
+   78.92%     0.00%  zbs-chunkd       libpthread-2.17.so
[.] __libc_recv




-- 
Thanks and Best Regards,
Feng Li(Alex)