netdev - Re: [PATCH RFC bpf-next 2/2] bpf: Use kmalloc_nolock() in local storage unconditionally

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAMB2axN8AajFkQUKpUwT6PrdHEa+CKUECBWC1dUQ=_67VBb4Eg@mail.gmail.com>
Date: Thu, 13 Nov 2025 10:58:49 -0800
From: Amery Hung <ameryhung@...il.com>
To: Alexei Starovoitov <alexei.starovoitov@...il.com>
Cc: bpf <bpf@...r.kernel.org>, Network Development <netdev@...r.kernel.org>, 
	Andrii Nakryiko <andrii@...nel.org>, Daniel Borkmann <daniel@...earbox.net>, 
	Martin KaFai Lau <martin.lau@...nel.org>, Kumar Kartikeya Dwivedi <memxor@...il.com>, KP Singh <kpsingh@...nel.org>, 
	Yonghong Song <yonghong.song@...ux.dev>, Song Liu <song@...nel.org>, 
	Kernel Team <kernel-team@...a.com>
Subject: Re: [PATCH RFC bpf-next 2/2] bpf: Use kmalloc_nolock() in local
 storage unconditionally

On Wed, Nov 12, 2025 at 5:08 PM Alexei Starovoitov
<alexei.starovoitov@...il.com> wrote:
>
> On Wed, Nov 12, 2025 at 1:15 PM Amery Hung <ameryhung@...il.com> wrote:
> >
> > On Wed, Nov 12, 2025 at 12:05 PM Alexei Starovoitov
> > <alexei.starovoitov@...il.com> wrote:
> > >
> > > On Wed, Nov 12, 2025 at 11:51 AM Amery Hung <ameryhung@...il.com> wrote:
> > > >
> > > > On Wed, Nov 12, 2025 at 11:35 AM Alexei Starovoitov
> > > > <alexei.starovoitov@...il.com> wrote:
> > > > >
> > > > > On Wed, Nov 12, 2025 at 9:59 AM Amery Hung <ameryhung@...il.com> wrote:
> > > > > >
> > > > > > @@ -80,23 +80,12 @@ bpf_selem_alloc(struct bpf_local_storage_map *smap, void *owner,
> > > > > >         if (mem_charge(smap, owner, smap->elem_size))
> > > > > >                 return NULL;
> > > > > >
> > > > > > -       if (smap->bpf_ma) {
> > > > > > -               selem = bpf_mem_cache_alloc_flags(&smap->selem_ma, gfp_flags);
> > > > > > -               if (selem)
> > > > > > -                       /* Keep the original bpf_map_kzalloc behavior
> > > > > > -                        * before started using the bpf_mem_cache_alloc.
> > > > > > -                        *
> > > > > > -                        * No need to use zero_map_value. The bpf_selem_free()
> > > > > > -                        * only does bpf_mem_cache_free when there is
> > > > > > -                        * no other bpf prog is using the selem.
> > > > > > -                        */
> > > > > > -                       memset(SDATA(selem)->data, 0, smap->map.value_size);
> > > > > > -       } else {
> > > > > > -               selem = bpf_map_kzalloc(&smap->map, smap->elem_size,
> > > > > > -                                       gfp_flags | __GFP_NOWARN);
> > > > > > -       }
> > > > > > +       selem = bpf_map_kmalloc_nolock(&smap->map, smap->elem_size, gfp_flags, NUMA_NO_NODE);
> > > > >
> > > > >
> > > > > Pls enable CONFIG_DEBUG_VM=y then you'll see that the above triggers:
> > > > > void *kmalloc_nolock_noprof(size_t size, gfp_t gfp_flags, int node)
> > > > > {
> > > > >         gfp_t alloc_gfp = __GFP_NOWARN | __GFP_NOMEMALLOC | gfp_flags;
> > > > > ...
> > > > >         VM_WARN_ON_ONCE(gfp_flags & ~(__GFP_ACCOUNT | __GFP_ZERO |
> > > > >                                       __GFP_NO_OBJ_EXT));
> > > > >
> > > > > and benchmarking numbers have to be redone, since with
> > > > > unsupported gfp flags kmalloc_nolock() is likely doing something wrong.
> > > >
> > > > I see. Thanks for pointing it out. Currently the verifier determines
> > > > the flag and rewrites the program based on if the caller of
> > > > storage_get helpers is sleepable. I will remove it and redo the
> > > > benchmark.
> > >
> > > yes. that part of the verifier can be removed too.
> > > First I would redo the benchmark numbers with s/gfp_flags/0 in the above line.
> >
> > Here are the new numbers after setting gfp_flags = __GFP_ZERO and
> > removing memset(0). BTW, the test is done on a physical machine. The
> > numbers for sk local storage can change ±1-2. I will try to increase
> > the test iteration or kill other unnecessary things running on the
> > machine to see if the number fluctuates less.
>
> fyi gfp_zero is pretty much the same memset().
> Shouldn't be any difference between __GFP_ZERO vs manual memset() after alloc.
>
> > Socket local storage
> > memory alloc     batch  creation speed              creation speed diff
> > ---------------  ----   ------------------                         ----
> > kzalloc           16    104.217 ± 0.974k/s  4.15 kmallocs/create
> > (before)          32    104.355 ± 0.606k/s  4.13 kmallocs/create
> >                   64    103.611 ± 0.707k/s  4.15 kmallocs/create
> >
> > kmalloc_nolock    16    100.402 ± 1.282k/s  1.11 kmallocs/create  -3.7%
> > (after)           32    101.592 ± 0.861k/s  1.07 kmallocs/create  -2.6%
> >                   64     98.995 ± 0.868k/s  1.07 kmallocs/create  -4.6%
>
> could you perf record/report both to see where the difference comes from?
> The only reason I could explain the difference is that kfree_nolock()
> is hitting defer_free() case, but that shouldn't happen
> in this microbenchmark. But if you see free_deferred_objects()
> in perf report that would explain it.

Updated numbers and next step:

A new perf result shows ~10% creation speed decrease (130k/s vs
142k/s) for socket local storage after stopping a service that is also
creating it for every connection.

The performance hit did come from defer_free() and the loss of
batching with kfree_rcu().

Hence, kmalloc_nolock() is not ready in this case to replace
kzalloc(). The next respin will only replace BPF memory allocator in
local storage with kmalloc_nolock().