linux-kernel - Re: [selftests/bpf] 69d96519db: kernel_selftests.bpf.test_socket

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Mon, 24 Jun 2019 14:24:22 -0700
From:   Stanislav Fomichev <sdf@...ichev.me>
To:     Andrii Nakryiko <andrii.nakryiko@...il.com>
Cc:     Rong Chen <rong.a.chen@...el.com>,
        Stanislav Fomichev <sdf@...gle.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Martin Lau <kafai@...com>, LKML <linux-kernel@...r.kernel.org>,
        Stephen Rothwell <sfr@...b.auug.org.au>,
        bpf <bpf@...r.kernel.org>, lkp@...org
Subject: Re: [selftests/bpf] 69d96519db:
 kernel_selftests.bpf.test_socket_cookie.fail

On 06/24, Andrii Nakryiko wrote:
> On Sun, Jun 23, 2019 at 5:59 PM Rong Chen <rong.a.chen@...el.com> wrote:
> >
> > On 6/22/19 6:27 AM, Stanislav Fomichev wrote:
> > > On 06/21, Andrii Nakryiko wrote:
> > >> )
> > >>
> > >> On Fri, Jun 21, 2019 at 9:11 AM Stanislav Fomichev <sdf@...ichev.me> wrote:
> > >>> On 06/21, kernel test robot wrote:
> > >>>> FYI, we noticed the following commit (built with gcc-7):
> > >>>>
> > >>>> commit: 69d96519dbf0bfa1868dc8597d4b9b2cdeb009d7 ("selftests/bpf: convert socket_cookie test to sk storage")
> > >>>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> > >>>>
> > >>>> in testcase: kernel_selftests
> > >>>> with following parameters:
> > >>>>
> > >>>>        group: kselftests-00
> > >>>>
> > >>>> test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
> > >>>> test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
> > >>>>
> > >>>>
> > >>>> on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
> > >>>>
> > >>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
> > >>>>
> > >>>>
> > >>>> If you fix the issue, kindly add following tag
> > >>>> Reported-by: kernel test robot <rong.a.chen@...el.com>
> > >>>>
> > >>>> # selftests: bpf: test_socket_cookie
> > >>>> # libbpf: failed to create map (name: 'socket_cookies'): Invalid
> > >>>> # argument
> > >>> Another case of old clang trying to create a map that depends on BTF?
> > >>> Should we maybe switch those BTF checks in the kernel to return
> > >>> EOPNOTSUPP to make it easy to diagnose?
> > >> For older compilers that don't generate DATASEC/VAR, you'll see a clear message:
> > >>
> > >> libbpf: DATASEC '.maps' not found.
> > >>
> > >> So this must be something else. I just confirmed with clang version
> > >> 7.0.20180201 that for ./test_socket_cookie that's the first line
> > >> that's emitted on failure.
> > > Thanks for checking, I also took a look at the attached kernel_selftests.xz,
> > > here is what it has:
> > > 2019-06-21 11:58:35 ln -sf /usr/bin/clang-6.0 /usr/bin/clang
> > > 2019-06-21 11:58:35 ln -sf /usr/bin/llc-6.0 /usr/bin/llc
> > > ...
> > > # BTF libbpf test[1] (test_btf_haskv.o): SKIP. No ELF .BTF found
> > > # BTF libbpf test[2] (test_btf_nokv.o): SKIP. No ELF .BTF found
> > > ...
> > > # Test case #0 (btf_dump_test_case_syntax): test_btf_dump_case:71:FAIL
> > > # failed to load test BTF: -2
> > > # Test case #1 (btf_dump_test_case_ordering): test_btf_dump_case:71:FAIL
> > > # failed to load test BTF: -2
> > > ...
> > >
> > > And so on. So there is clearly an old clang that doesn't emit any
> > > BTF. And I also don't see your recent abd29c931459 before 69d96519dbf0 in
> > > linux-next, that's why it doesn't complain about missing/corrupt BTF.
> 
> Ah, ok, that would explain it. But in any case, clang 6&7 is too old.
> Clang 8 or better yet clang 9 (for global data, datasec/var-dependent
> stuff) would be great.
While we are it: I think I have resolved the BTF story internally,
so if you want to go ahead and convert the rest of the tests to
BTF format, I would not object anymore ;-)

(I didn't expect it to be that easy initially, so sorry if I wasted
everyones time arguing about it).

> > >
> > > We need to convince lkp people to upgrade clang, otherwise, I suppose,
> > > we'll get more of these reportings after your recent df0b77925982 :-(
> >
> > Thanks for the clarification, we'll upgrade clang asap.
> 
> Thanks Rong!
> 
> >
> > Best Regards,
> > Rong Chen
> >
> >
> > >
> > >>>> # libbpf: failed to load object './socket_cookie_prog.o'
> > >>>> # (test_socket_cookie.c:149: errno: Invalid argument) Failed to load
> > >>>> # ./socket_cookie_prog.o
> > >>>> # FAILED
> > >>>> not ok 15 selftests: bpf: test_socket_cookie
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> To reproduce:
> > >>>>
> > >>>>          # build kernel
> > >>>>        cd linux
> > >>>>        cp config-5.2.0-rc2-00598-g69d9651 .config
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
> > >>>>
> > >>>>
> > >>>>          git clone https://github.com/intel/lkp-tests.git
> > >>>>          cd lkp-tests
> > >>>>          bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
> > >>>>
> > >>>>
> > >>>>
> > >>>> Thanks,
> > >>>> Rong Chen
> > >>>>
> > >> <mega snip>