netdev - Re: [PATCH RFC bpf-next 2/2] bpf: Use kmalloc_nolock() in local storage unconditionally

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMB2axOEauSyi13-acjDJUB--8+V7ZUG+r2V=fATwZHDQjFH4w@mail.gmail.com>
Date: Wed, 12 Nov 2025 13:14:51 -0800
From: Amery Hung <ameryhung@...il.com>
To: Alexei Starovoitov <alexei.starovoitov@...il.com>
Cc: bpf <bpf@...r.kernel.org>, Network Development <netdev@...r.kernel.org>, 
	Andrii Nakryiko <andrii@...nel.org>, Daniel Borkmann <daniel@...earbox.net>, 
	Martin KaFai Lau <martin.lau@...nel.org>, Kumar Kartikeya Dwivedi <memxor@...il.com>, KP Singh <kpsingh@...nel.org>, 
	Yonghong Song <yonghong.song@...ux.dev>, Song Liu <song@...nel.org>, 
	Kernel Team <kernel-team@...a.com>
Subject: Re: [PATCH RFC bpf-next 2/2] bpf: Use kmalloc_nolock() in local
 storage unconditionally

On Wed, Nov 12, 2025 at 12:05 PM Alexei Starovoitov
<alexei.starovoitov@...il.com> wrote:
>
> On Wed, Nov 12, 2025 at 11:51 AM Amery Hung <ameryhung@...il.com> wrote:
> >
> > On Wed, Nov 12, 2025 at 11:35 AM Alexei Starovoitov
> > <alexei.starovoitov@...il.com> wrote:
> > >
> > > On Wed, Nov 12, 2025 at 9:59 AM Amery Hung <ameryhung@...il.com> wrote:
> > > >
> > > > @@ -80,23 +80,12 @@ bpf_selem_alloc(struct bpf_local_storage_map *smap, void *owner,
> > > >         if (mem_charge(smap, owner, smap->elem_size))
> > > >                 return NULL;
> > > >
> > > > -       if (smap->bpf_ma) {
> > > > -               selem = bpf_mem_cache_alloc_flags(&smap->selem_ma, gfp_flags);
> > > > -               if (selem)
> > > > -                       /* Keep the original bpf_map_kzalloc behavior
> > > > -                        * before started using the bpf_mem_cache_alloc.
> > > > -                        *
> > > > -                        * No need to use zero_map_value. The bpf_selem_free()
> > > > -                        * only does bpf_mem_cache_free when there is
> > > > -                        * no other bpf prog is using the selem.
> > > > -                        */
> > > > -                       memset(SDATA(selem)->data, 0, smap->map.value_size);
> > > > -       } else {
> > > > -               selem = bpf_map_kzalloc(&smap->map, smap->elem_size,
> > > > -                                       gfp_flags | __GFP_NOWARN);
> > > > -       }
> > > > +       selem = bpf_map_kmalloc_nolock(&smap->map, smap->elem_size, gfp_flags, NUMA_NO_NODE);
> > >
> > >
> > > Pls enable CONFIG_DEBUG_VM=y then you'll see that the above triggers:
> > > void *kmalloc_nolock_noprof(size_t size, gfp_t gfp_flags, int node)
> > > {
> > >         gfp_t alloc_gfp = __GFP_NOWARN | __GFP_NOMEMALLOC | gfp_flags;
> > > ...
> > >         VM_WARN_ON_ONCE(gfp_flags & ~(__GFP_ACCOUNT | __GFP_ZERO |
> > >                                       __GFP_NO_OBJ_EXT));
> > >
> > > and benchmarking numbers have to be redone, since with
> > > unsupported gfp flags kmalloc_nolock() is likely doing something wrong.
> >
> > I see. Thanks for pointing it out. Currently the verifier determines
> > the flag and rewrites the program based on if the caller of
> > storage_get helpers is sleepable. I will remove it and redo the
> > benchmark.
>
> yes. that part of the verifier can be removed too.
> First I would redo the benchmark numbers with s/gfp_flags/0 in the above line.

Here are the new numbers after setting gfp_flags = __GFP_ZERO and
removing memset(0). BTW, the test is done on a physical machine. The
numbers for sk local storage can change ±1-2. I will try to increase
the test iteration or kill other unnecessary things running on the
machine to see if the number fluctuates less.

Socket local storage
memory alloc     batch  creation speed              creation speed diff
---------------  ----   ------------------                         ----
kzalloc           16    104.217 ± 0.974k/s  4.15 kmallocs/create
(before)          32    104.355 ± 0.606k/s  4.13 kmallocs/create
                  64    103.611 ± 0.707k/s  4.15 kmallocs/create

kmalloc_nolock    16    100.402 ± 1.282k/s  1.11 kmallocs/create  -3.7%
(after)           32    101.592 ± 0.861k/s  1.07 kmallocs/create  -2.6%
                  64     98.995 ± 0.868k/s  1.07 kmallocs/create  -4.6%

Task local storage
memory alloc     batch  creation speed              creation speed diff
---------------  ----   ------------------                         ----
BPF memory        16     24.668 ± 0.121k/s  2.54 kmallocs/create
allocator         32     22.899 ± 0.097k/s  2.67 kmallocs/create
(before)          64     22.559 ± 0.076k/s  2.56 kmallocs/create

kmalloc_nolock    16     25.659 ± 0.108k/s  2.51 kmallocs/create  +4.0%
(after)           32     23.723 ± 0.082k/s  2.54 kmallocs/create  +3.6%
                  64     23.359 ± 0.236k/s  2.48 kmallocs/create  +3.5%