linux-kernel - Re: [PATCH 2/2] btf: Add the option to include global variable types

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAEf4BzYvFnqeZjNy_b_VP9DEpBaTMWuMAau8j6ZAWtgwcE5ysg@mail.gmail.com>
Date: Tue, 25 Feb 2025 13:52:30 -0800
From: Andrii Nakryiko <andrii.nakryiko@...il.com>
To: Alan Maguire <alan.maguire@...cle.com>
Cc: Alexei Starovoitov <alexei.starovoitov@...il.com>, 
	Stephen Brennan <stephen.s.brennan@...cle.com>, Masahiro Yamada <masahiroy@...nel.org>, 
	Andrii Nakryiko <andrii@...nel.org>, Nicolas Schier <nicolas@...sle.eu>, Kees Cook <kees@...nel.org>, 
	KP Singh <kpsingh@...nel.org>, Martin KaFai Lau <martin.lau@...ux.dev>, 
	Sami Tolvanen <samitolvanen@...gle.com>, Eduard Zingerman <eddyz87@...il.com>, 
	linux-arch <linux-arch@...r.kernel.org>, Stanislav Fomichev <sdf@...ichev.me>, 
	Kent Overstreet <kent.overstreet@...ux.dev>, Pasha Tatashin <pasha.tatashin@...een.com>, 
	Jiri Olsa <jolsa@...nel.org>, John Fastabend <john.fastabend@...il.com>, 
	Jann Horn <jannh@...gle.com>, Ard Biesheuvel <ardb@...nel.org>, 
	Yonghong Song <yonghong.song@...ux.dev>, Hao Luo <haoluo@...gle.com>, 
	Andrew Morton <akpm@...ux-foundation.org>, 
	Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>, Daniel Borkmann <daniel@...earbox.net>, 
	Arnd Bergmann <arnd@...db.de>, Nathan Chancellor <nathan@...nel.org>, linux-debuggers@...r.kernel.org, 
	Alexei Starovoitov <ast@...nel.org>, Song Liu <song@...nel.org>, LKML <linux-kernel@...r.kernel.org>, 
	bpf <bpf@...r.kernel.org>
Subject: Re: [PATCH 2/2] btf: Add the option to include global variable types

On Tue, Feb 25, 2025 at 2:02 AM Alan Maguire <alan.maguire@...cle.com> wrote:
>
> On 07/02/2025 23:50, Alexei Starovoitov wrote:
> > On Thu, Feb 6, 2025 at 5:21 PM Stephen Brennan
> > <stephen.s.brennan@...cle.com> wrote:
> >> When the feature was implemented in pahole, my measurements indicated
> >> that vmlinux BTF size increased by about 25.8%, and module BTF size
> >> increased by 53.2%. Due to these increases, the feature is implemented
> >> behind a new config option, allowing users sensitive to increased memory
> >> usage to disable it.
> >>
> >
> > ...
> >> +config DEBUG_INFO_BTF_GLOBAL_VARS
> >> +       bool "Generate BTF type information for all global variables"
> >> +       default y
> >> +       depends on DEBUG_INFO_BTF && PAHOLE_VERSION >= 128
> >> +       help
> >> +         Include type information for all global variables in the BTF. This
> >> +         increases the size of the BTF information, which increases memory
> >> +         usage at runtime. With global variable types available, runtime
> >> +         debugging and tracers may be able to provide more detail.
> >
> > This is not a solution.
> > Even if it's changed to 'default n' distros will enable it
> > like they enable everything and will suffer a regression.
> >
> > We need to add a new module like vmlinux_btf.ko that will contain
> > this additional BTF data. For global vars and everything else we might need.
> >
>
> In this area, I've been exploring adding support for
> CONFIG_DEBUG_INFO_BTF=m , so that the BTF info for vmlinux is delivered
> via a module. From the consumer side, everything looks identical
> (/sys/kernel/btf/vmlinux is there etc), it is just that the .BTF section
> is delivered via btf_vmlinux.ko instead. The original need for this was
> that embedded folks noted that because in the current situation BTF data
> is in vmlinux, they cannot enable BTF because such small-footprint
> systems do not support a large vmlinux binary. However they could
> potentially use kernel BTF if it was delivered via a module. The other
> nice thing about module delivery in the general case is we can make use
> of module compression. In experiments I see a 5.8Mb vmlinux BTF reduce
> to a 1.8Mb btf_vmlinux.ko.gz module on-disk.
>
> The challenge in delivering vmlinux BTF in a module is that on module
> load during boot other modules expect vmlinux BTF to be there when
> adding their own BTF to /sys/kernel/btf. And kfunc registration from
> kernel and modules expects this also. So support for deferred BTF module
> load/kfunc registration is required too. I've implemented the former and
> now am working on the latter. Hope to have some RFC patches ready soon,
> but it looks feasible at this point.

Lazy btf_vmlinux.ko loading when BTF is actually needed (i.e., when
user reads /sys/kernel/btf/vmlinux for the first time; or when BPF
program is validated and needs kernel BTF) would be great. Curious too
see how all that fits together!

>
> Assuming such an option was available to small-footprint systems, should
> we consider adding global variables to core vmlinux BTF along with
> per-cpu variables? Then vmlinux BTF extras could be used for some of the
> additional optional representations like function site-specific data
> (inlines etc)? Or are there other factors other than on-disk footprint
> that we need to consider? Thanks!

I'd keep BTF for variables separate from "core" vmlinux BTF. We can
have /sys/kernel/btf/vmlinux.vars, which would depend on
/sys/kernel/btf/vmlinux as a base BTF. Separately, we could eventually
have /sys/kernel/btf/vmlinux.inlines which would also have
/sys/kernel/btf/vmlinux as base BTF. If no one needs vmlinux.vars on
the system, we won't need to waste memory on it. Seems more modular
and extensible.

>
> Alan
>
> > pw-bot: cr
> >
>