Date:   Sat, 18 Jun 2022 18:43:06 +0200
From:   Jörn-Thorben Hinz <jthinz@...lbox.tu-berlin.de>
To:     Martin KaFai Lau <kafai@...com>
CC:     <bpf@...r.kernel.org>, Alexei Starovoitov <ast@...nel.org>,
        "Daniel Borkmann" <daniel@...earbox.net>,
        Andrii Nakryiko <andrii@...nel.org>, <netdev@...r.kernel.org>
Subject: Re: [PATCH bpf-next v3 3/5] selftests/bpf: Test a BPF CC writing
 sk_pacing_*

On Fri, 2022-06-17 at 14:04 -0700, Martin KaFai Lau wrote:
> On Tue, Jun 14, 2022 at 12:44:50PM +0200, Jörn-Thorben Hinz wrote:
> > Test whether a TCP CC implemented in BPF is allowed to write
> > sk_pacing_rate and sk_pacing_status in struct sock. This is needed
> > when
> > cong_control() is implemented and used.
> > 
> > Signed-off-by: Jörn-Thorben Hinz <jthinz@...lbox.tu-berlin.de>
> > ---
> >  .../selftests/bpf/prog_tests/bpf_tcp_ca.c     | 21 +++++++
> >  .../bpf/progs/tcp_ca_write_sk_pacing.c        | 60
> > +++++++++++++++++++
> >  2 files changed, 81 insertions(+)
> >  create mode 100644
> > tools/testing/selftests/bpf/progs/tcp_ca_write_sk_pacing.c
> > 
> > diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
> > b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
> > index e9a9a31b2ffe..a797497e2864 100644
> > --- a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
> > +++ b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c
> > @@ -9,6 +9,7 @@
> >  #include "bpf_cubic.skel.h"
> >  #include "bpf_tcp_nogpl.skel.h"
> >  #include "bpf_dctcp_release.skel.h"
> > +#include "tcp_ca_write_sk_pacing.skel.h"
> >  
> >  #ifndef ENOTSUPP
> >  #define ENOTSUPP 524
> > @@ -322,6 +323,24 @@ static void test_rel_setsockopt(void)
> >         bpf_dctcp_release__destroy(rel_skel);
> >  }
> >  
> > +static void test_write_sk_pacing(void)
> > +{
> > +       struct tcp_ca_write_sk_pacing *skel;
> > +       struct bpf_link *link;
> > +
> > +       skel = tcp_ca_write_sk_pacing__open_and_load();
> > +       if (!ASSERT_OK_PTR(skel, "open_and_load")) {
> nit. Remove this single line '{'.
> 
> ./scripts/checkpatch.pl has reported that also:
> WARNING: braces {} are not necessary for single statement blocks
> #43: FILE: tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c:332:
> +       if (!ASSERT_OK_PTR(skel, "open_and_load")) {
> +               return;
> +       }
Have to admit I knowingly disregarded that warning as more of a
recommendation. Out of habit and since I personally don’t see any
compelling reason to generally use single-line statements after ifs,
only multiple disadvantages.

But wrong place to argue here, of course. Will bow to the warning.

> 
> 
> > +               return;
> > +       }
> > +
> > +       link = bpf_map__attach_struct_ops(skel-
> > >maps.write_sk_pacing);
> > +       if (ASSERT_OK_PTR(link, "attach_struct_ops")) {
> Same here.
> 
> and no need to check the link before bpf_link__destroy.
> bpf_link__destroy can handle error link.  Something like:
> 
>         ASSERT_OK_PTR(link, "attach_struct_ops");
>         bpf_link__destroy(link);
>         tcp_ca_write_sk_pacing__destroy(skel);
> 
> The earlier examples in test_cubic and test_dctcp were
> written before bpf_link__destroy can handle error link.
You are right, I followed the other two test_*() functions there. Good
to know that it behaves similar to (k)free() and others. Will remove
the ifs around bpf_link__destroy().

> 
> > +               bpf_link__destroy(link);
> > +       }
> > +
> > +       tcp_ca_write_sk_pacing__destroy(skel);
> > +}
> > +
> >  void test_bpf_tcp_ca(void)
> >  {
> >         if (test__start_subtest("dctcp"))
> > @@ -334,4 +353,6 @@ void test_bpf_tcp_ca(void)
> >                 test_dctcp_fallback();
> >         if (test__start_subtest("rel_setsockopt"))
> >                 test_rel_setsockopt();
> > +       if (test__start_subtest("write_sk_pacing"))
> > +               test_write_sk_pacing();
> >  }
> > diff --git
> > a/tools/testing/selftests/bpf/progs/tcp_ca_write_sk_pacing.c
> > b/tools/testing/selftests/bpf/progs/tcp_ca_write_sk_pacing.c
> > new file mode 100644
> > index 000000000000..43447704cf0e
> > --- /dev/null
> > +++ b/tools/testing/selftests/bpf/progs/tcp_ca_write_sk_pacing.c
> > @@ -0,0 +1,60 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include "vmlinux.h"
> > +
> > +#include <bpf/bpf_helpers.h>
> > +#include <bpf/bpf_tracing.h>
> > +
> > +char _license[] SEC("license") = "GPL";
> > +
> > +#define USEC_PER_SEC 1000000UL
> > +
> > +#define min(a, b) ((a) < (b) ? (a) : (b))
> > +
> > +static inline struct tcp_sock *tcp_sk(const struct sock *sk)
> > +{
> This helper is already available in bpf_tcp_helpers.h.
> Is there a reason not to use that one and redefine
> it in both patch 3 and 4?  The mss_cache and srtt_us can be added
> to bpf_tcp_helpers.h.  It will need another effort to move
> all selftest's bpf-cc to vmlinux.h.
I fully agree it’s not elegant to redefine tcp_sk() twice more.

It was between either using bpf_tcp_helpers.h and adding and
maintaining additional struct members there. Or using the (as I
understood it) more “modern” approach with vmlinux.h and redefining the
trivial tcp_sk(). I chose the later. Didn’t see a reason not to slowly
introduce vmlinux.h into the CA tests.

I had the same dilemma for the algorithm I’m implementing: Reuse
bpf_tcp_helpers.h from the kernel tree and extend it. Or use vmlinux.h
and copy only some of the (mostly trivial) helper functions. Also chose
the later here.

While doing the above, I also considered extracting the type
declarations from bpf_tcp_helpers.h into an, e.g.,
bpf_tcp_types_helper.h, keeping only the functions in
bpf_tcp_helpers.h. bpf_tcp_helpers.h could have been a base helper for
any BPF CA implementation then and used with either vmlinux.h or the
“old-school” includes. Similar to the way bpf_helpers.h is used. But at
that point, a bpf_tcp_types_helper.h could have probably just been
dropped for good and in favor of vmlinux.h. So I didn’t continue with
that.

Do you insist to use bpf_tcp_helpers.h instead of vmlinux.h? Or could
the described split into two headers make sense after all?

(Will wait for your reply here before sending a v4.)

> 
> > +       return (struct tcp_sock *)sk;
> > +}
> > +
> > +SEC("struct_ops/write_sk_pacing_init")
> > +void BPF_PROG(write_sk_pacing_init, struct sock *sk)
> > +{
> > +#ifdef ENABLE_ATOMICS_TESTS
> > +       __sync_bool_compare_and_swap(&sk->sk_pacing_status,
> > SK_PACING_NONE,
> > +                                    SK_PACING_NEEDED);
> > +#else
> > +       sk->sk_pacing_status = SK_PACING_NEEDED;
> > +#endif
> > +}
> > +
> > +SEC("struct_ops/write_sk_pacing_cong_control")
> > +void BPF_PROG(write_sk_pacing_cong_control, struct sock *sk,
> > +             const struct rate_sample *rs)
> > +{
> > +       const struct tcp_sock *tp = tcp_sk(sk);
> > +       unsigned long rate =
> > +               ((tp->snd_cwnd * tp->mss_cache * USEC_PER_SEC) <<
> > 3) /
> > +               (tp->srtt_us ?: 1U << 3);
> > +       sk->sk_pacing_rate = min(rate, sk->sk_max_pacing_rate);
> > +}
> > +
> > +SEC("struct_ops/write_sk_pacing_ssthresh")
> > +__u32 BPF_PROG(write_sk_pacing_ssthresh, struct sock *sk)
> > +{
> > +       return tcp_sk(sk)->snd_ssthresh;
> > +}
> > +
> > +SEC("struct_ops/write_sk_pacing_undo_cwnd")
> > +__u32 BPF_PROG(write_sk_pacing_undo_cwnd, struct sock *sk)
> > +{
> > +       return tcp_sk(sk)->snd_cwnd;
> > +}
> > +
> > +SEC(".struct_ops")
> > +struct tcp_congestion_ops write_sk_pacing = {
> > +       .init = (void *)write_sk_pacing_init,
> > +       .cong_control = (void *)write_sk_pacing_cong_control,
> > +       .ssthresh = (void *)write_sk_pacing_ssthresh,
> > +       .undo_cwnd = (void *)write_sk_pacing_undo_cwnd,
> > +       .name = "bpf_w_sk_pacing",
> > +};
> > -- 
> > 2.30.2
> > 

Powered by blists - more mailing lists