netdev - Re: [PATCH bpf-next v2] libbpf: don't allocate 16M for log buffer by default

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAEf4BzbEuWdb-77nm=o5doGfYbpWxbTe+U2mM+KH1hw6CnYuww@mail.gmail.com>
Date:   Wed, 25 Mar 2020 12:06:06 -0700
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Stanislav Fomichev <sdf@...gle.com>
Cc:     Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>,
        "David S. Miller" <davem@...emloft.net>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>
Subject: Re: [PATCH bpf-next v2] libbpf: don't allocate 16M for log buffer by default

On Wed, Mar 25, 2020 at 9:20 AM Stanislav Fomichev <sdf@...gle.com> wrote:
>
> For each prog/btf load we allocate and free 16 megs of verifier buffer.
> On production systems it doesn't really make sense because the
> programs/btf have gone through extensive testing and (mostly) guaranteed
> to successfully load.
>
> Let's assume successful case by default and skip buffer allocation
> on the first try. If there is an error, start with BPF_LOG_BUF_SIZE
> and double it on each ENOSPC iteration.
>
> v2:
> * Don't allocate the buffer at all on the first try (Andrii Nakryiko)
>
> Signed-off-by: Stanislav Fomichev <sdf@...gle.com>
> ---
>  tools/lib/bpf/btf.c    | 20 +++++++++++++++-----
>  tools/lib/bpf/libbpf.c | 22 ++++++++++++++--------
>  2 files changed, 29 insertions(+), 13 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 3d1c25fc97ae..bfef3d606b54 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -657,22 +657,32 @@ int btf__finalize_data(struct bpf_object *obj, struct btf *btf)
>
>  int btf__load(struct btf *btf)
>  {
> -       __u32 log_buf_size = BPF_LOG_BUF_SIZE;
> +       __u32 log_buf_size = 0;
>         char *log_buf = NULL;
>         int err = 0;
>
>         if (btf->fd >= 0)
>                 return -EEXIST;
>
> -       log_buf = malloc(log_buf_size);
> -       if (!log_buf)
> -               return -ENOMEM;
> +retry_load:
> +       if (log_buf_size) {
> +               log_buf = malloc(log_buf_size);
> +               if (!log_buf)
> +                       return -ENOMEM;
>
> -       *log_buf = 0;
> +               *log_buf = 0;
> +       }
>
>         btf->fd = bpf_load_btf(btf->data, btf->data_size,
>                                log_buf, log_buf_size, false);
>         if (btf->fd < 0) {
> +               if (!log_buf || errno == ENOSPC) {
> +                       log_buf_size = max((__u32)BPF_LOG_BUF_SIZE,
> +                                          log_buf_size << 1);
> +                       free(log_buf);
> +                       goto retry_load;
> +               }
> +
>                 err = -errno;
>                 pr_warn("Error loading BTF: %s(%d)\n", strerror(errno), errno);
>                 if (*log_buf)
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 085e41f9b68e..b2fd43a03708 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -4855,8 +4855,8 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
>  {
>         struct bpf_load_program_attr load_attr;
>         char *cp, errmsg[STRERR_BUFSIZE];
> -       int log_buf_size = BPF_LOG_BUF_SIZE;
> -       char *log_buf;
> +       size_t log_buf_size = 0;
> +       char *log_buf = NULL;
>         int btf_fd, ret;
>
>         if (!insns || !insns_cnt)
> @@ -4896,22 +4896,28 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
>         load_attr.prog_flags = prog->prog_flags;
>
>  retry_load:
> -       log_buf = malloc(log_buf_size);
> -       if (!log_buf)
> -               pr_warn("Alloc log buffer for bpf loader error, continue without log\n");
> +       if (log_buf_size) {
> +               log_buf = malloc(log_buf_size);
> +               if (!log_buf)
> +                       pr_warn("Alloc log buffer for bpf loader error, continue without log\n");
> +

considering that log_buf is not allocated first time, if it fails to
allocate on retry then it should be an error, no?

Otherwise looks good, please add my ack after fixing this:

Acked-by: Andrii Nakryiko <andriin@...com>

> +               *log_buf = 0;
> +       }
>
>         ret = bpf_load_program_xattr(&load_attr, log_buf, log_buf_size);
>
>         if (ret >= 0) {
> -               if (load_attr.log_level)
> +               if (log_buf && load_attr.log_level)
>                         pr_debug("verifier log:\n%s", log_buf);
>                 *pfd = ret;
>                 ret = 0;
>                 goto out;
>         }
>
> -       if (errno == ENOSPC) {
> -               log_buf_size <<= 1;
> +       if (!log_buf || errno == ENOSPC) {
> +               log_buf_size = max((size_t)BPF_LOG_BUF_SIZE,
> +                                  log_buf_size << 1);
> +
>                 free(log_buf);
>                 goto retry_load;
>         }
> --
> 2.25.1.696.g5e7596f4ac-goog
>