netdev - Re: [PATCH] selftests: Skip BPF seftests by default

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <94010653-0cb3-d804-7410-a571480d6db2@linuxfoundation.org>
Date:   Thu, 17 Dec 2020 08:53:36 -0700
From:   Shuah Khan <skhan@...uxfoundation.org>
To:     Mark Brown <broonie@...nel.org>,
        Seth Forshee <seth.forshee@...onical.com>
Cc:     Shuah Khan <shuah@...nel.org>, Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Andrii Nakryiko <andrii@...nel.org>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...nel.org>,
        Daniel Diaz <daniel.diaz@...aro.org>,
        Veronika Kabatova <vkabatov@...hat.com>,
        netdev@...r.kernel.org, bpf@...r.kernel.org,
        linux-kselftest@...r.kernel.org,
        Guillaume Tucker <guillaume.tucker@...labora.com>,
        Kevin Hilman <khilman@...libre.com>,
        Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] selftests: Skip BPF seftests by default

On 12/17/20 6:07 AM, Mark Brown wrote:
> On Wed, Dec 16, 2020 at 04:05:58PM -0600, Seth Forshee wrote:
>> On Thu, Dec 10, 2020 at 06:52:33PM +0000, Mark Brown wrote:
> 
>>> as part of the wider kselftest build by specifying SKIP_TARGETS,
>>> including setting an empty SKIP_TARGETS to build everything.  They can
>>> also continue to build the BPF selftests individually in cases where
>>> they are specifically focused on BPF.
> 
>> Why not just remove the line which adds bpf to TARGETS? This has the
>> same effect, but doesn't require an emtpy SKIP_TARGETS to run them. We
>> have testing scripts which use 'make TARGETS=bpf ...' which will have to
>> be updated, and I doubt we are the only ones.
> 

I would prefer leaving bpf in the main Makefile TARGETS. This will be
useful to users that have their systems setup for bpf builds.

>> I also feel like this creates confusing semantics around SKIP_TARGETS.
>> If I don't supply a value then I don't get the bpf selftests, but then
>> if I try to use SKIP_TARGETS to skip some other test suddenly I do get
>> them. That's counterintuitive.
> 
> That's what I did first, it's also messy just differently.  If you
> don't add bpf to TARGETS then if you do what's needed to get it building
> it becomes inconvenient to run it as part of running everything else at
> the top level since you need to enumerate all the targets.  It felt like
> skipping is what we're actually doing here and it seems like those
> actively working with BPF will be used to having to update things in
> their environment.  People who start using SKIP_TARGETS are *probably*
> going to find out about it from the Makefile anyway so will see the
> default that's there.
> 
> Fundamentally having such demanding build dependencies is always going
> to result in some kind of mess, it's just where we push it.
> 
>> I also wanted to point out that the net/test_bpf.sh selftest requires
>> having the test_bpf module from the bpf selftest build. So when the bpf
>> selftests aren't built this test is guaranteed to fail. Though it would
>> be nice if the net selftests didn't require building the bpf self tests
>> in order to pass.
> 
> Right, that's a separate issue - the net tests should really skip that
> if they don't have BPF, as we do for other runtime detectable
> dependencies.  It's nowhere near as severe as failing to build though.
> 

Correct. This has to be handled as a run-time dependency check and skip
instead of fail.

thanks,
-- Shuah