Date:   Wed, 3 Jan 2018 12:58:26 +0800
From:   "Wangnan (F)" <wangnan0@...wei.com>
To:     Arnaldo Carvalho de Melo <acme@...nel.org>
CC:     Adrian Hunter <adrian.hunter@...el.com>,
        David Ahern <dsahern@...il.com>, Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        "Linux Kernel Mailing List" <linux-kernel@...r.kernel.org>
Subject: Re: perf test BPF failing on 4.15.0-rc6

And please check if the kprobe created by

  $ perf probe -v SyS_epoll_wait

works for the test program used by this testcase:

#include <sys/epoll.h>
#include <stdlib.h>

#define NR_ITERS 100

static int epoll_wait_loop(void)
{
         int i;

         /* Should fail NR_ITERS times */
         for (i = 0; i < NR_ITERS; i++)
                 epoll_wait(-(i + 1), NULL, 0, 0);
         return 0;
}

int main()
{
         epoll_wait_loop();
         return 0;
}

In my configuration it gives 100 samples:

$ sudo ./perf record -e probe:SyS_epoll_wait /tmp/a.out
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.016 MB perf.data (100 samples) ]

I guess in your configuration you would get 3 tracepoints, and can get 0 
sample from them.

Thank you.

On 2018/1/3 12:42, Wangnan (F) wrote:
> Unable to reproduce. In my kernel configuration, SyS_epoll_wait is not 
> inlined at all.
>
> From the log you sent, it seems that all 3 instances are attached. 
> This testcase should work if the last one (SyS_epoll_wait) get probed 
> correctly.
>
> Could you please have a look if the 3rd kprobe event 
> (p:perf_bpf_probe/func_2 _text+2845824) works?
>
> Thank you.
>
> On 2018/1/3 2:59, Arnaldo Carvalho de Melo wrote:
>> Hi Wang,
>>
>>     I just updated my machine to Fedora 27 and 4.15.0-rc6 and the
>> only test failing for me is:
>>
>> [root@...et linux]# perf test bpf
>> 39: BPF filter                                            :
>> 39.1: Basic BPF filtering                                 : FAILED!
>> 39.2: BPF pinning                                         : Skip
>> 39.3: BPF prologue generation                             : Skip
>> 39.4: BPF relocation checker                              : Skip
>> [root@...et linux]#
>>
>> I haven't checked but perhaps the problem is that SyS_epoll_wait seems
>> to now be inlined in three places and perhaps the eBPF proggie is being
>> added to just one of them?
>>
>> Seemingly relevant excerpt:
>>
>> Open Debuginfo file: /lib/modules/4.15.0-rc6/build/vmlinux
>> Try to find probe point from debuginfo.
>> Matched function: SyS_epoll_wait [2f40eb7]
>> found inline addr: 0xffffffff812b6ff1
>> Probe point found: compat_SyS_epoll_pwait+129
>> found inline addr: 0xffffffff812b6de7
>> Probe point found: SyS_epoll_pwait+135
>> found inline addr: 0xffffffff812b6c80
>> Probe point found: SyS_epoll_wait+0
>> Found 3 probe_trace_events.
>>
>> - Arnaldo
>>
>> P.S.: Full -v output:
>>
>> [root@...et linux]# uname -a
>> Linux jouet 4.15.0-rc6 #4 SMP Tue Jan 2 14:30:53 -03 2018 x86_64 
>> x86_64 x86_64 GNU/Linux
>> [root@...et linux]# gcc -v
>> Using built-in specs.
>> COLLECT_GCC=/usr/bin/gcc
>> COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/7/lto-wrapper
>> OFFLOAD_TARGET_NAMES=nvptx-none
>> OFFLOAD_TARGET_DEFAULT=1
>> Target: x86_64-redhat-linux
>> Configured with: ../configure --enable-bootstrap 
>> --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto 
>> --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info 
>> --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared 
>> --enable-threads=posix --enable-checking=release --enable-multilib 
>> --with-system-zlib --enable-__cxa_atexit 
>> --disable-libunwind-exceptions --enable-gnu-unique-object 
>> --enable-linker-build-id --with-gcc-major-version-only 
>> --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array 
>> --with-isl --enable-libmpx --enable-offload-targets=nvptx-none 
>> --without-cuda-driver --enable-gnu-indirect-function 
>> --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
>> Thread model: posix
>> gcc version 7.2.1 20170915 (Red Hat 7.2.1-2) (GCC)
>> [root@...et linux]# clang -v
>> clang version 6.0.0 (http://llvm.org/git/clang.git 
>> 56cc8f8880db2ebc433eeb6b6a707c101467a186) 
>> (http://llvm.org/git/llvm.git 3656d83960a4f3fedf6d8f19043abf52379f78c3)
>> Target: x86_64-unknown-linux-gnu
>> Thread model: posix
>> InstalledDir: /usr/local/bin
>> Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
>> Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
>> Candidate multilib: .;@m64
>> Candidate multilib: 32;@m32
>> Selected multilib: .;@m64
>> [root@...et linux]# perf test -v BPF
>> 39: BPF filter                                            :
>> 39.1: Basic BPF filtering                                 :
>> --- start ---
>> test child forked, pid 24304
>> Kernel build dir is set to /lib/modules/4.15.0-rc6/build
>> set env: KBUILD_DIR=/lib/modules/4.15.0-rc6/build
>> unset env: KBUILD_OPTS
>> include option is set to  -nostdinc -isystem 
>> /usr/lib/gcc/x86_64-redhat-linux/7/include 
>> -I/home/acme/git/linux/arch/x86/include 
>> -I./arch/x86/include/generated  -I/home/acme/git/linux/include 
>> -I./include -I/home/acme/git/linux/arch/x86/include/uapi 
>> -I./arch/x86/include/generated/uapi 
>> -I/home/acme/git/linux/include/uapi -I./include/generated/uapi 
>> -include /home/acme/git/linux/include/linux/kconfig.h
>> set env: NR_CPUS=4
>> set env: LINUX_VERSION_CODE=0x40f00
>> set env: CLANG_EXEC=/usr/local/bin/clang
>> set env: CLANG_OPTIONS=-xc
>> set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
>> /usr/lib/gcc/x86_64-redhat-linux/7/include 
>> -I/home/acme/git/linux/arch/x86/include 
>> -I./arch/x86/include/generated  -I/home/acme/git/linux/include 
>> -I./include -I/home/acme/git/linux/arch/x86/include/uapi 
>> -I./arch/x86/include/generated/uapi 
>> -I/home/acme/git/linux/include/uapi -I./include/generated/uapi 
>> -include /home/acme/git/linux/include/linux/kconfig.h
>> set env: WORKING_DIR=/lib/modules/4.15.0-rc6/build
>> set env: CLANG_SOURCE=-
>> llvm compiling command template: echo '/*
>>   * bpf-script-example.c
>>   * Test basic LLVM building
>>   */
>> #ifndef LINUX_VERSION_CODE
>> # error Need LINUX_VERSION_CODE
>> # error Example: for 4.2 kernel, put 
>> 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of 
>> ~/.perfconfig'
>> #endif
>> #define BPF_ANY 0
>> #define BPF_MAP_TYPE_ARRAY 2
>> #define BPF_FUNC_map_lookup_elem 1
>> #define BPF_FUNC_map_update_elem 2
>>
>> static void *(*bpf_map_lookup_elem)(void *map, void *key) =
>>     (void *) BPF_FUNC_map_lookup_elem;
>> static void *(*bpf_map_update_elem)(void *map, void *key, void 
>> *value, int flags) =
>>     (void *) BPF_FUNC_map_update_elem;
>>
>> struct bpf_map_def {
>>     unsigned int type;
>>     unsigned int key_size;
>>     unsigned int value_size;
>>     unsigned int max_entries;
>> };
>>
>> #define SEC(NAME) __attribute__((section(NAME), used))
>> struct bpf_map_def SEC("maps") flip_table = {
>>     .type = BPF_MAP_TYPE_ARRAY,
>>     .key_size = sizeof(int),
>>     .value_size = sizeof(int),
>>     .max_entries = 1,
>> };
>>
>> SEC("func=SyS_epoll_wait")
>> int bpf_func__SyS_epoll_wait(void *ctx)
>> {
>>     int ind =0;
>>     int *flag = bpf_map_lookup_elem(&flip_table, &ind);
>>     int new_flag;
>>     if (!flag)
>>         return 0;
>>     /* flip flag and store back */
>>     new_flag = !*flag;
>>     bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
>>     return new_flag;
>> }
>> char _license[] SEC("license") = "GPL";
>> int _version SEC("version") = LINUX_VERSION_CODE;
>> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS 
>> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS 
>> $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign 
>> -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o -
>> libbpf: loading object '[basic_bpf_test]' from buffer
>> libbpf: section .strtab, size 120, link 0, flags 0, type=3
>> libbpf: section .text, size 0, link 0, flags 6, type=1
>> libbpf: section func=SyS_epoll_wait, size 192, link 0, flags 6, type=1
>> libbpf: found program func=SyS_epoll_wait
>> libbpf: section .relfunc=SyS_epoll_wait, size 32, link 8, flags 0, 
>> type=9
>> libbpf: section maps, size 16, link 0, flags 3, type=1
>> libbpf: section license, size 4, link 0, flags 3, type=1
>> libbpf: license of [basic_bpf_test] is GPL
>> libbpf: section version, size 4, link 0, flags 3, type=1
>> libbpf: kernel version of [basic_bpf_test] is 40f00
>> libbpf: section .symtab, size 168, link 1, flags 0, type=2
>> libbpf: maps in [basic_bpf_test]: 1 maps in 16 bytes
>> libbpf: map 0 is "flip_table"
>> libbpf: collecting relocating info for: 'func=SyS_epoll_wait'
>> libbpf: relocation: insn_idx=4
>> libbpf: relocation: find map 0 (flip_table) for insn 4
>> libbpf: relocation: insn_idx=17
>> libbpf: relocation: find map 0 (flip_table) for insn 17
>> bpf: config program 'func=SyS_epoll_wait'
>> symbol:SyS_epoll_wait file:(null) line:0 offset:0 return:0 lazy:(null)
>> bpf: config 'func=SyS_epoll_wait' is ok
>> Looking at the vmlinux_path (8 entries long)
>> Using /lib/modules/4.15.0-rc6/build/vmlinux for symbols
>> Open Debuginfo file: /lib/modules/4.15.0-rc6/build/vmlinux
>> Try to find probe point from debuginfo.
>> Matched function: SyS_epoll_wait [2f40eb7]
>> found inline addr: 0xffffffff812b6ff1
>> Probe point found: compat_SyS_epoll_pwait+129
>> found inline addr: 0xffffffff812b6de7
>> Probe point found: SyS_epoll_pwait+135
>> found inline addr: 0xffffffff812b6c80
>> Probe point found: SyS_epoll_wait+0
>> Found 3 probe_trace_events.
>> Opening /sys/kernel/debug/tracing//kprobe_events write=1
>> Writing event: p:perf_bpf_probe/func _text+2846705
>> Writing event: p:perf_bpf_probe/func_1 _text+2846183
>> Writing event: p:perf_bpf_probe/func_2 _text+2845824
>> libbpf: create map flip_table: fd=3
>> add bpf event perf_bpf_probe:func and attach bpf program 4
>> registering plugin: /root/.traceevent/plugins/plugin_kvm.so
>> registering plugin: /root/.traceevent/plugins/plugin_hrtimer.so
>> registering plugin: /root/.traceevent/plugins/plugin_jbd2.so
>> registering plugin: /root/.traceevent/plugins/plugin_kmem.so
>> registering plugin: /root/.traceevent/plugins/plugin_scsi.so
>> registering plugin: /root/.traceevent/plugins/plugin_sched_switch.so
>> registering plugin: /root/.traceevent/plugins/plugin_function.so
>> registering plugin: /root/.traceevent/plugins/plugin_xen.so
>> registering plugin: /root/.traceevent/plugins/plugin_mac80211.so
>> registering plugin: /root/.traceevent/plugins/plugin_cfg80211.so
>> adding perf_bpf_probe:func
>> adding perf_bpf_probe:func to 0x24e3850
>> add bpf event perf_bpf_probe:func_1 and attach bpf program 4
>> adding perf_bpf_probe:func_1
>> adding perf_bpf_probe:func_1 to 0x1965020
>> add bpf event perf_bpf_probe:func_2 and attach bpf program 4
>> adding perf_bpf_probe:func_2
>> adding perf_bpf_probe:func_2 to 0x19656a0
>> mmap size 1052672B
>> BPF filter result incorrect, expected 56, got 0 samples
>> Opening /sys/kernel/debug/tracing//kprobe_events write=1
>> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>> Parsing probe_events: p:perf_bpf_probe/func _text+2846705
>> Group:perf_bpf_probe Event:func probe:p
>> Parsing probe_events: p:perf_bpf_probe/func_1 _text+2846183
>> Group:perf_bpf_probe Event:func_1 probe:p
>> Parsing probe_events: p:perf_bpf_probe/func_2 _text+2845824
>> Group:perf_bpf_probe Event:func_2 probe:p
>> Writing event: -:perf_bpf_probe/func
>> Opening /sys/kernel/debug/tracing//kprobe_events write=1
>> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>> Parsing probe_events: p:perf_bpf_probe/func_1 _text+2846183
>> Group:perf_bpf_probe Event:func_1 probe:p
>> Parsing probe_events: p:perf_bpf_probe/func_2 _text+2845824
>> Group:perf_bpf_probe Event:func_2 probe:p
>> Writing event: -:perf_bpf_probe/func_1
>> Opening /sys/kernel/debug/tracing//kprobe_events write=1
>> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>> Parsing probe_events: p:perf_bpf_probe/func_2 _text+2845824
>> Group:perf_bpf_probe Event:func_2 probe:p
>> Writing event: -:perf_bpf_probe/func_2
>> test child finished with -1
>> ---- end ----
>> BPF filter subtest 0: FAILED!
>> 39.2: BPF pinning                                         :
>> --- force skipped ---
>> BPF filter subtest 1: Skip
>> 39.3: BPF prologue generation                             :
>> --- force skipped ---
>> BPF filter subtest 2: Skip
>> 39.4: BPF relocation checker                              :
>> --- force skipped ---
>> BPF filter subtest 3: Skip
>> [root@...et linux]#
>> [root@...et linux]#
>

Powered by blists - more mailing lists