netdev - Re: [PATCH bpf-next 1/2] selftests/bpf: skip verifier tests that depend on CONFIG_CGROUP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <de121723-9298-ba9f-8092-d23dd9d8403e@netronome.com>
Date:   Thu, 13 Dec 2018 17:02:48 +0000
From:   Quentin Monnet <quentin.monnet@...ronome.com>
To:     Stanislav Fomichev <sdf@...ichev.me>
Cc:     Daniel Borkmann <daniel@...earbox.net>,
        Alexei Starovoitov <alexei.starovoitov@...il.com>,
        Edward Cree <ecree@...arflare.com>,
        Stanislav Fomichev <sdf@...gle.com>, netdev@...r.kernel.org,
        davem@...emloft.net, ast@...nel.org
Subject: Re: [PATCH bpf-next 1/2] selftests/bpf: skip verifier tests that
 depend on CONFIG_CGROUP_BPF

2018-12-13 08:37 UTC-0800 ~ Stanislav Fomichev <sdf@...ichev.me>
> On 12/13, Quentin Monnet wrote:
>> 2018-12-13 12:52 UTC+0100 ~ Daniel Borkmann <daniel@...earbox.net>
>>> On 12/13/2018 07:06 AM, Alexei Starovoitov wrote:
>>>> On Wed, Dec 12, 2018 at 02:32:01PM -0800, Stanislav Fomichev wrote:
>>>>>
>>>>> To summarize, I like your idea about doing runtime tests and I think I
>>>>> can make it work quite nicely without any config_disabled ugliness by
>>>>> looking at the prog_type of each test.
>>>>> I can send an RFC patch series out if there still a small chance you could
>>>>> take it, but if you've already set you mind, I'd just keep them
>>>>> internally. So let me know if you have a hard NACK on the runtime probing
>>>>> approach or there is still some wiggle room.
>>>>
>>>> If there is no uapi/bpf.h change, it's likely fine.
>>>> Like if test_verifier tries to load 'foo() {return 0;}' prog
>>>> for the .prog_type in the test that failed to confirm that
>>>> such prog type is supported by the kernel...
>>>> that is fine, since no extra prog_loads are happening for the default case.
>>>
>>> I think this would kind of go along the lines of what Quentin is working on.
>>> Idea [0] is to consolidate effort into bpftool so that one can do something
>>> like `bpftool kernel probe` and it generates a header file with CONFIG_*
>>> defines for features where bpftool was able to successfully probe the
>>> underlying kernel with. This would allow developers to include this header
>>> generation as part of the build workflow and avoid having to implement
>>> similar probing mechanism in various projects over and over again which aim
>>> to run on different kernel versions. I'm wondering whether it would make sense
>>> to split the probing part and put it into libbpf where then bpftool is only
>>> responsible to call the API and write out the defines? That way, the runtime
>>> probing could potentially be reused for selftests as well?
> +1
> 
> Keeping those low-level probing details in the libbpf seems like a
> good idea. `bpftool feature` can then be just a simple a frontend to those
> probes to dump them in C/JSON. Tests and other tools can use the
> probes on the target host via libbpf to degrade some functionality or
> print nice error messages instead of 'EINVAL: Invalid argument'.
> 
>>>
>>> Thanks,
>>> Daniel
>>>
>>>  [0] slide 8,12: http://vger.kernel.org/lpc_bpf2018_talks/qmo-bpf-slides-v2.pdf
>>
>> Hi Daniel, Stanislav,
>>
>> Thanks for the Cc. I got somewhat delayed in my series, but I just
>> finished it and was about to post the patches to the mailing list. Since
>> the code is ready to go I'll send it in its current shape, i.e. all
>> probes implemented on bpftool side, and we can hopefully use it as a
>> support for further discussion.
> Just out of curiosity: what's the usecase of generating C defines via
> bpftool? Is it for the BCC case where we have the complier on the target
> host and can run bpftool+bcc there?

Yes, this is the idea. It produces a header file that one can include at
compile time in a project: bcc is a candidate, Cilium would be another
(see [0], [1]), for example. As mentioned by Daniel, I suspect a growing
number of projects will need this kind of probing, so having bpftool
able to do the work and to dump the result as macros could avoid
implementing multiple versions of the same thing.

[0] https://github.com/cilium/cilium/blob/master/bpf/run_probes.sh
[1] https://github.com/cilium/cilium/tree/master/bpf/probes