Message-ID: <20181213171022.GI9107@mini-arch.hsd1.ca.comcast.net>
Date:   Thu, 13 Dec 2018 09:10:22 -0800
From:   Stanislav Fomichev <sdf@...ichev.me>
To:     Quentin Monnet <quentin.monnet@...ronome.com>
Cc:     Daniel Borkmann <daniel@...earbox.net>,
        Alexei Starovoitov <alexei.starovoitov@...il.com>,
        Edward Cree <ecree@...arflare.com>,
        Stanislav Fomichev <sdf@...gle.com>, netdev@...r.kernel.org,
        davem@...emloft.net, ast@...nel.org
Subject: Re: [PATCH bpf-next 1/2] selftests/bpf: skip verifier tests that
 depend on CONFIG_CGROUP_BPF

On 12/13, Quentin Monnet wrote:
> 2018-12-13 08:37 UTC-0800 ~ Stanislav Fomichev <sdf@...ichev.me>
> > On 12/13, Quentin Monnet wrote:
> >> 2018-12-13 12:52 UTC+0100 ~ Daniel Borkmann <daniel@...earbox.net>
> >>> On 12/13/2018 07:06 AM, Alexei Starovoitov wrote:
> >>>> On Wed, Dec 12, 2018 at 02:32:01PM -0800, Stanislav Fomichev wrote:
> >>>>>
> >>>>> To summarize, I like your idea about doing runtime tests and I think I
> >>>>> can make it work quite nicely without any config_disabled ugliness by
> >>>>> looking at the prog_type of each test.
> >>>>> I can send an RFC patch series out if there still a small chance you could
> >>>>> take it, but if you've already set you mind, I'd just keep them
> >>>>> internally. So let me know if you have a hard NACK on the runtime probing
> >>>>> approach or there is still some wiggle room.
> >>>>
> >>>> If there is no uapi/bpf.h change, it's likely fine.
> >>>> Like if test_verifier tries to load 'foo() {return 0;}' prog
> >>>> for the .prog_type in the test that failed to confirm that
> >>>> such prog type is supported by the kernel...
> >>>> that is fine, since no extra prog_loads are happening for the default case.
> >>>
> >>> I think this would kind of go along the lines of what Quentin is working on.
> >>> Idea [0] is to consolidate effort into bpftool so that one can do something
> >>> like `bpftool kernel probe` and it generates a header file with CONFIG_*
> >>> defines for features where bpftool was able to successfully probe the
> >>> underlying kernel with. This would allow developers to include this header
> >>> generation as part of the build workflow and avoid having to implement
> >>> similar probing mechanism in various projects over and over again which aim
> >>> to run on different kernel versions. I'm wondering whether it would make sense
> >>> to split the probing part and put it into libbpf where then bpftool is only
> >>> responsible to call the API and write out the defines? That way, the runtime
> >>> probing could potentially be reused for selftests as well?
> > +1
> > 
> > Keeping those low-level probing details in the libbpf seems like a
> > good idea. `bpftool feature` can then be just a simple a frontend to those
> > probes to dump them in C/JSON. Tests and other tools can use the
> > probes on the target host via libbpf to degrade some functionality or
> > print nice error messages instead of 'EINVAL: Invalid argument'.
> > 
> >>>
> >>> Thanks,
> >>> Daniel
> >>>
> >>>  [0] slide 8,12: http://vger.kernel.org/lpc_bpf2018_talks/qmo-bpf-slides-v2.pdf
> >>
> >> Hi Daniel, Stanislav,
> >>
> >> Thanks for the Cc. I got somewhat delayed in my series, but I just
> >> finished it and was about to post the patches to the mailing list. Since
> >> the code is ready to go I'll send it in its current shape, i.e. all
> >> probes implemented on bpftool side, and we can hopefully use it as a
> >> support for further discussion.
> > Just out of curiosity: what's the usecase of generating C defines via
> > bpftool? Is it for the BCC case where we have the complier on the target
> > host and can run bpftool+bcc there?
> 
> Yes, this is the idea. It produces a header file that one can include at
> compile time in a project: bcc is a candidate, Cilium would be another
> (see [0], [1]), for example. As mentioned by Daniel, I suspect a growing
> number of projects will need this kind of probing, so having bpftool
> able to do the work and to dump the result as macros could avoid
> implementing multiple versions of the same thing.
I'm also interested in a usecase where we don't have a compiler on the
target, but runtime should still be able to proble some features and
maybe refuse the service or degrade to a different program/feature.
Those are the same probes, but executed dynamically on the target and I
think having them (maybe not all?) in the libbpf is useful. WDYT?

> 
> [0] https://github.com/cilium/cilium/blob/master/bpf/run_probes.sh
> [1] https://github.com/cilium/cilium/tree/master/bpf/probes

Powered by blists - more mailing lists