| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <aQR8if0cpQQ5Am36@tiehlicka>
Date: Fri, 31 Oct 2025 10:08:25 +0100
From: Michal Hocko <mhocko@...e.com>
To: Roman Gushchin <roman.gushchin@...ux.dev>
Cc: Andrew Morton <akpm@...ux-foundation.org>, linux-kernel@...r.kernel.org,
Alexei Starovoitov <ast@...nel.org>,
Suren Baghdasaryan <surenb@...gle.com>,
Shakeel Butt <shakeel.butt@...ux.dev>,
Johannes Weiner <hannes@...xchg.org>,
Andrii Nakryiko <andrii@...nel.org>,
JP Kobryn <inwardvessel@...il.com>, linux-mm@...ck.org,
cgroups@...r.kernel.org, bpf@...r.kernel.org,
Martin KaFai Lau <martin.lau@...nel.org>,
Song Liu <song@...nel.org>,
Kumar Kartikeya Dwivedi <memxor@...il.com>,
Tejun Heo <tj@...nel.org>
Subject: Re: [PATCH v2 10/23] mm: introduce BPF kfuncs to access memcg
statistics and events
On Mon 27-10-25 16:17:13, Roman Gushchin wrote:
> Introduce BPF kfuncs to conveniently access memcg data:
> - bpf_mem_cgroup_vm_events(),
> - bpf_mem_cgroup_usage(),
> - bpf_mem_cgroup_page_state(),
> - bpf_mem_cgroup_flush_stats().
>
> These functions are useful for implementing BPF OOM policies, but
> also can be used to accelerate access to the memcg data. Reading
> it through cgroupfs is much more expensive, roughly 5x, mostly
> because of the need to convert the data into the text and back.
>
> JP Kobryn:
> An experiment was setup to compare the performance of a program that
> uses the traditional method of reading memory.stat vs a program using
> the new kfuncs. The control program opens up the root memory.stat file
> and for 1M iterations reads, converts the string values to numeric data,
> then seeks back to the beginning. The experimental program sets up the
> requisite libbpf objects and for 1M iterations invokes a bpf program
> which uses the kfuncs to fetch all available stats for node_stat_item,
> memcg_stat_item, and vm_event_item types.
>
> The results showed a significant perf benefit on the experimental side,
> outperforming the control side by a margin of 93%. In kernel mode,
> elapsed time was reduced by 80%, while in user mode, over 99% of time
> was saved.
>
> control: elapsed time
> real 0m38.318s
> user 0m25.131s
> sys 0m13.070s
>
> experiment: elapsed time
> real 0m2.789s
> user 0m0.187s
> sys 0m2.512s
>
> control: perf data
> 33.43% a.out libc.so.6 [.] __vfscanf_internal
> 6.88% a.out [kernel.kallsyms] [k] vsnprintf
> 6.33% a.out libc.so.6 [.] _IO_fgets
> 5.51% a.out [kernel.kallsyms] [k] format_decode
> 4.31% a.out libc.so.6 [.] __GI_____strtoull_l_internal
> 3.78% a.out [kernel.kallsyms] [k] string
> 3.53% a.out [kernel.kallsyms] [k] number
> 2.71% a.out libc.so.6 [.] _IO_sputbackc
> 2.41% a.out [kernel.kallsyms] [k] strlen
> 1.98% a.out a.out [.] main
> 1.70% a.out libc.so.6 [.] _IO_getline_info
> 1.51% a.out libc.so.6 [.] __isoc99_sscanf
> 1.47% a.out [kernel.kallsyms] [k] memory_stat_format
> 1.47% a.out [kernel.kallsyms] [k] memcpy_orig
> 1.41% a.out [kernel.kallsyms] [k] seq_buf_printf
>
> experiment: perf data
> 10.55% memcgstat bpf_prog_..._query [k] bpf_prog_16aab2f19fa982a7_query
> 6.90% memcgstat [kernel.kallsyms] [k] memcg_page_state_output
> 3.55% memcgstat [kernel.kallsyms] [k] _raw_spin_lock
> 3.12% memcgstat [kernel.kallsyms] [k] memcg_events
> 2.87% memcgstat [kernel.kallsyms] [k] __memcg_slab_post_alloc_hook
> 2.73% memcgstat [kernel.kallsyms] [k] kmem_cache_free
> 2.70% memcgstat [kernel.kallsyms] [k] entry_SYSRETQ_unsafe_stack
> 2.25% memcgstat [kernel.kallsyms] [k] __memcg_slab_free_hook
> 2.06% memcgstat [kernel.kallsyms] [k] get_page_from_freelist
>
> Signed-off-by: Roman Gushchin <roman.gushchin@...ux.dev>
> Co-developed-by: JP Kobryn <inwardvessel@...il.com>
> Signed-off-by: JP Kobryn <inwardvessel@...il.com>
Acked-by: Michal Hocko <mhocko@...e.com>
> ---
> include/linux/memcontrol.h | 2 ++
> mm/bpf_memcontrol.c | 57 +++++++++++++++++++++++++++++++++++++-
> 2 files changed, 58 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 39a6c7c8735b..b9e08dddd7ad 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -953,6 +953,8 @@ static inline void mod_memcg_page_state(struct page *page,
> rcu_read_unlock();
> }
>
> +unsigned long memcg_events(struct mem_cgroup *memcg, int event);
> +unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap);
> unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx);
> unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item);
> unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx);
> diff --git a/mm/bpf_memcontrol.c b/mm/bpf_memcontrol.c
> index 76c342318256..387255b8ab88 100644
> --- a/mm/bpf_memcontrol.c
> +++ b/mm/bpf_memcontrol.c
> @@ -75,6 +75,56 @@ __bpf_kfunc void bpf_put_mem_cgroup(struct mem_cgroup *memcg)
> css_put(&memcg->css);
> }
>
> +/**
> + * bpf_mem_cgroup_vm_events - Read memory cgroup's vm event counter
> + * @memcg: memory cgroup
> + * @event: event id
> + *
> + * Allows to read memory cgroup event counters.
> + */
> +__bpf_kfunc unsigned long bpf_mem_cgroup_vm_events(struct mem_cgroup *memcg,
> + enum vm_event_item event)
> +{
> + return memcg_events(memcg, event);
> +}
> +
> +/**
> + * bpf_mem_cgroup_usage - Read memory cgroup's usage
> + * @memcg: memory cgroup
> + *
> + * Returns current memory cgroup size in bytes.
> + */
> +__bpf_kfunc unsigned long bpf_mem_cgroup_usage(struct mem_cgroup *memcg)
> +{
> + return page_counter_read(&memcg->memory);
> +}
> +
> +/**
> + * bpf_mem_cgroup_page_state - Read memory cgroup's page state counter
> + * @memcg: memory cgroup
> + * @idx: counter idx
> + *
> + * Allows to read memory cgroup statistics. The output is in bytes.
> + */
> +__bpf_kfunc unsigned long bpf_mem_cgroup_page_state(struct mem_cgroup *memcg, int idx)
> +{
> + if (idx < 0 || idx >= MEMCG_NR_STAT)
> + return (unsigned long)-1;
> +
> + return memcg_page_state_output(memcg, idx);
> +}
> +
> +/**
> + * bpf_mem_cgroup_flush_stats - Flush memory cgroup's statistics
> + * @memcg: memory cgroup
> + *
> + * Propagate memory cgroup's statistics up the cgroup tree.
> + */
> +__bpf_kfunc void bpf_mem_cgroup_flush_stats(struct mem_cgroup *memcg)
> +{
> + mem_cgroup_flush_stats(memcg);
> +}
> +
> __bpf_kfunc_end_defs();
>
> BTF_KFUNCS_START(bpf_memcontrol_kfuncs)
> @@ -82,6 +132,11 @@ BTF_ID_FLAGS(func, bpf_get_root_mem_cgroup, KF_ACQUIRE | KF_RET_NULL)
> BTF_ID_FLAGS(func, bpf_get_mem_cgroup, KF_ACQUIRE | KF_RET_NULL | KF_RCU)
> BTF_ID_FLAGS(func, bpf_put_mem_cgroup, KF_RELEASE)
>
> +BTF_ID_FLAGS(func, bpf_mem_cgroup_vm_events, KF_TRUSTED_ARGS)
> +BTF_ID_FLAGS(func, bpf_mem_cgroup_usage, KF_TRUSTED_ARGS)
> +BTF_ID_FLAGS(func, bpf_mem_cgroup_page_state, KF_TRUSTED_ARGS)
> +BTF_ID_FLAGS(func, bpf_mem_cgroup_flush_stats, KF_TRUSTED_ARGS | KF_SLEEPABLE)
> +
> BTF_KFUNCS_END(bpf_memcontrol_kfuncs)
>
> static const struct btf_kfunc_id_set bpf_memcontrol_kfunc_set = {
> @@ -93,7 +148,7 @@ static int __init bpf_memcontrol_init(void)
> {
> int err;
>
> - err = register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS,
> + err = register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC,
> &bpf_memcontrol_kfunc_set);
> if (err)
> pr_warn("error while registering bpf memcontrol kfuncs: %d", err);
> --
> 2.51.0
--
Michal Hocko
SUSE Labs
Powered by blists - more mailing lists