Message-Id: <49AF84D3-3D47-4B35-B1A7-497045FD241F@dtucker.co.uk>
Date: Fri, 10 Jan 2025 16:22:24 +0000
From: Dave Tucker <dave@...cker.co.uk>
To: Arnaldo Carvalho de Melo <acme@...nel.org>
Cc: Tamir Duberstein <tamird@...il.com>, Alice Ryhl <aliceryhl@...gle.com>,
 Neal Gompa <neal@...pa.dev>,
 Miguel Ojeda <miguel.ojeda.sandonis@...il.com>,
 Matthew Maurer <mmaurer@...gle.com>, Miguel Ojeda <ojeda@...nel.org>,
 Alex Gaynor <alex.gaynor@...il.com>, Boqun Feng <boqun.feng@...il.com>,
 Gary Guo <gary@...yguo.net>,
 Björn Roy Baron <bjorn3_gh@...tonmail.com>,
 Benno Lossin <benno.lossin@...ton.me>,
 Andreas Hindborg <a.hindborg@...nel.org>,
 Trevor Gross <tmgross@...ch.edu>, linux-kernel@...r.kernel.org,
 rust-for-linux@...r.kernel.org, Matthias Maennich <maennich@...gle.com>,
 bpf <bpf@...r.kernel.org>, Martin KaFai Lau <martin.lau@...ux.dev>,
 Alexei Starovoitov <ast@...nel.org>,
 Daniel Borkmann <daniel@...earbox.net>,
 Andrii Nakryiko <andrii@...nel.org>,
 John Fastabend <john.fastabend@...il.com>,
 Eric Curtin <ecurtin@...hat.com>, Martin Reboredo <yakoyoku@...il.com>,
 Alessandro Decina <alessandro.d@...il.com>,
 Michal Rostecki <vadorovsky@...tonmail.com>
Subject: Re: [PATCH] rust: Disallow BTF generation with Rust + LTO



> On 9 Jan 2025, at 22:39, Arnaldo Carvalho de Melo <acme@...nel.org> wrote:
> 
> On Thu, Jan 09, 2025 at 01:29:47PM -0300, Arnaldo Carvalho de Melo wrote:
>> On Thu, Jan 09, 2025 at 10:49:49AM -0500, Tamir Duberstein wrote:
>>> On Thu, Jan 9, 2025 at 10:47 AM Arnaldo Carvalho de Melo <acme@...nel.org> wrote:
>>>> I was thinking about it after reading this thread yesterday, i.e. we
>>>> could encode constructs from Rust that can be represented in BTF and
>>>> skip the ones that can't, pruning types that depend on non BTF
>>>> representable types, etc.
>> 
>>> Yep, this is what bpf-linker does, along with some other things[0]. I
>>> highly recommend reading the code I linked to avoid re-discovering
>>> these things.
>> 
>> Sure, thanks for pointing it out and suggest I read it while
>> experimenting with having the same concept in pahole, I'll try a quick
>> hack and then look at it to see how close I got to what you guys came up
>> with :-)
> 
> So I didn't manage to work on this today, just this quick hack:
> 
> ⬢ [acme@...lbox pahole]$ git diff
> diff --git a/btf_encoder.c b/btf_encoder.c
> index 78efd705333e2e52..5610e0902f2cd347 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -1559,7 +1559,7 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag,
>        default:
>                fprintf(stderr, "Unsupported DW_TAG_%s(0x%x): type: 0x%x\n",
>                        dwarf_tag_name(tag->tag), tag->tag, ref_type_id);
> -               return -1;
> +               return 0;
>        }
> }
> 
> ⬢ [acme@...lbox pahole]$
> 
> Which essentially encodes any DWARF tag that the BTF encoder doesn't
> know about into 'void'.
> 
> Super quick hack, I still have to look at the implications, but some
> results:
> 
> ⬢ [acme@...lbox pahole]$ cp ../build/rust-kernel/vmlinux vmlinux.rust
> ⬢ [acme@...lbox pahole]$ pahole --btf_encode vmlinux.rust 
> die__process_class: tag not supported 0x33 (variant_part) at <4c9c589>!
> die__create_new_enumeration: DW_TAG_subprogram (0x2e) @ <0x4cb784b> not handled in a rust CU!
> tag__recode_dwarf_type: couldn't find name for function 0x4cd72bf, abstract_origin=0, specification=0x4cb784b
> ⬢ [acme@...lbox pahole]$ 
> ⬢ [acme@...lbox pahole]$ pahole -F btf vmlinux.rust | less
> ⬢ [acme@...lbox pahole]$ 
> ⬢ [acme@...lbox pahole]$ 
> ⬢ [acme@...lbox pahole]$ pahole -F btf vmlinux.rust | less
> ⬢ [acme@...lbox pahole]$ 
> ⬢ [acme@...lbox pahole]$ bpftool btf dump file vmlinux.rust | less
> ⬢ [acme@...lbox pahole]$ bpftool btf dump file vmlinux.rust | head
> [1] INT 'DW_ATE_signed_32' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> [2] INT 'DW_ATE_signed_64' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED
> [3] INT 'DW_ATE_unsigned_1' size=1 bits_offset=0 nr_bits=8 encoding=(none)
> [4] INT 'DW_ATE_unsigned_8' size=1 bits_offset=0 nr_bits=8 encoding=(none)
> [5] INT 'DW_ATE_unsigned_64' size=8 bits_offset=0 nr_bits=64 encoding=(none)
> [6] INT 'DW_ATE_unsigned_32' size=4 bits_offset=0 nr_bits=32 encoding=(none)
> [7] STRUCT 'tracepoint' size=80 vlen=9
> 'name' type_id=8 bits_offset=0
> 'key' type_id=12 bits_offset=64
> 'static_call_key' type_id=23 bits_offset=192
> ⬢ [acme@...lbox pahole]$ bpftool btf dump file vmlinux.rust | tail
> type_id=12300 offset=226816 size=24 (VAR 'cfd_data')
> type_id=12301 offset=226880 size=8 (VAR 'call_single_queue')
> type_id=12302 offset=226944 size=32 (VAR 'csd_data')
> type_id=51838 offset=227008 size=832 (VAR 'softnet_data')
> type_id=54492 offset=227840 size=24 (VAR 'rt_uncached_list')
> type_id=55984 offset=227904 size=24 (VAR 'rt6_uncached_list')
> type_id=8094 offset=229376 size=64 (VAR 'vmw_steal_time')
> type_id=8810 offset=229440 size=64 (VAR 'apf_reason')
> type_id=8811 offset=229504 size=64 (VAR 'steal_time')
> type_id=8813 offset=229568 size=8 (VAR 'kvm_apic_eoi')
> ⬢ [acme@...lbox pahole]$ readelf -wi vmlinux.rust | grep DW_AT_producer | head
>    <d>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <1be5a>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <1ec5b>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <3bfd5>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <3d11c>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <4fcf0>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <503d5>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <5c067>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <5c42e>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
>    <5efd9>   DW_AT_producer    : (indexed string: 0): clang version 18.1.6 (Fedora 18.1.6-3.fc40)
> ⬢ [acme@...lbox pahole]$ readelf -wi vmlinux.rust | grep DW_AT_lang | grep -i rust
>    <4c91de1>   DW_AT_language    : 28 (Rust)
>    <4ce66ea>   DW_AT_language    : 28 (Rust)
>    <4cf2cfa>   DW_AT_language    : 28 (Rust)
>    <4cf71a8>   DW_AT_language    : 28 (Rust)
>    <4d2797d>   DW_AT_language    : 28 (Rust)
>    <4d50f34>   DW_AT_language    : 28 (Rust)
> ⬢ [acme@...lbox pahole]$
> 
>  Compilation Unit @ offset 0x4c91dd1:
>   Length:        0x54905 (32-bit)
>   Version:       4
>   Abbrev Offset: 0x244258
>   Pointer Size:  8
> <0><4c91ddc>: Abbrev Number: 1 (DW_TAG_compile_unit)
>    <4c91ddd>   DW_AT_producer    : (indirect string, offset: 0x282d41): clang LLVM (rustc version 1.80.0 (051478957 2024-07-21) (Fedora 1.80.0-1.fc40))
>    <4c91de1>   DW_AT_language    : 28  (Rust)
>    <4c91de3>   DW_AT_name        : (indirect string, offset: 0x282d91): /usr/lib/rustlib/src/rust/library/core/src/lib.rs/@...re.3f32dfd9e3bca37e-cgu.0
>    <4c91de7>   DW_AT_stmt_list   : 0xa1d85f
>    <4c91deb>   DW_AT_comp_dir    : (indirect string, offset: 0x2ebc6): /home/acme/git/build/rust-kernel
>    <4c91def>   DW_AT_low_pc      : 0
>    <4c91df7>   DW_AT_ranges      : 0x1e5f0
> <1><4c91dfb>: Abbrev Number: 2 (DW_TAG_variable)
>    <4c91dfc>   DW_AT_name        : (indirect string, offset: 0x555b60): <usize as core::fmt::Debug>::{vtable}
>    <4c91e00>   DW_AT_type        : <0x4c91e0e>
>    <4c91e04>   DW_AT_location    : 9 byte block: 3 78 34 50 82 ff ff ff ff     (DW_OP_addr: ffffffff82503478)
> <1><4c91e0e>: Abbrev Number: 3 (DW_TAG_structure_type)
>    <4c91e0f>   DW_AT_containing_type: <0x4c91e5a>
>    <4c91e13>   DW_AT_name        : (indirect string, offset: 0x2b3380): <usize as core::fmt::Debug>::{vtable_type}
>    <4c91e17>   DW_AT_byte_size   : 32
>    <4c91e18>   DW_AT_alignment   : 8
> <2><4c91e19>: Abbrev Number: 4 (DW_TAG_member)
>    <4c91e1a>   DW_AT_name        : (indirect string, offset: 0x313232): drop_in_place
>    <4c91e1e>   DW_AT_type        : <0x4c91e46>
>    <4c91e22>   DW_AT_alignment   : 8
>    <4c91e23>   DW_AT_data_member_location: 0
> <2><4c91e24>: Abbrev Number: 4 (DW_TAG_member)
>    <4c91e25>   DW_AT_name        : (indirect string, offset: 0x570b7e): size
>    <4c91e29>   DW_AT_type        : <0x4c91e5a>
> 
> ⬢ [acme@...lbox pahole]$ pahole -F btf -C "<usize as core::fmt::Debug>::{vtable_type}" vmlinux.rust 
> struct <usize as core::fmt::Debug>::{vtable_type} {
> __SANITIZED_FAKE_INT__ *   drop_in_place;        /*     0     8 */
> usize                      size;                 /*     8     8 */
> usize                      align;                /*    16     8 */
> __SANITIZED_FAKE_INT__ *   __method3;            /*    24     8 */
> 
> /* size: 32, cachelines: 1, members: 4 */
> /* last cacheline: 32 bytes */
> };
> 
> ⬢ [acme@...lbox pahole]$
> 
> ⬢ [acme@...lbox pahole]$ pahole -F btf -C '<core::fmt::Error as core::fmt::Debug>::{vtable_type}' vmlinux.rust 
> struct <core::fmt::Error as core::fmt::Debug>::{vtable_type} {
> __SANITIZED_FAKE_INT__ *   drop_in_place;        /*     0     8 */
> usize                      size;                 /*     8     8 */
> usize                      align;                /*    16     8 */
> __SANITIZED_FAKE_INT__ *   __method3;            /*    24     8 */
> 
> /* size: 32, cachelines: 1, members: 4 */
> /* last cacheline: 32 bytes */
> };
> 
> ⬢ [acme@...lbox pahole]$
> 
> ⬢ [acme@...lbox pahole]$ pahole --show_decl_info -F dwarf -C Alignment vmlinux.rust 
> die__process_class: tag not supported 0x33 (variant_part) at <4c9c589>!
> die__create_new_enumeration: DW_TAG_subprogram (0x2e) @ <0x4cb784b> not handled in a rust CU!
> tag__recode_dwarf_type: couldn't find name for function 0x4cd72bf, abstract_origin=0, specification=0x4cb784b
> /* Used at: /usr/lib/rustlib/src/rust/library/core/src/lib.rs/@...re.3f32dfd9e3bca37e-cgu.0 */
> /* <4c9c50c> (null):32530 */
> enum Alignment {
> Left    = 0,
> Right   = 1,
> Center  = 2,
> Unknown = 3,
> } __attribute__((__packed__));
> 
> ⬢ [acme@...lbox pahole]$ pahole --show_decl_info -F btf -C Alignment vmlinux.rust 
> /* Used at: vmlinux.rust */
> /* <0> (null):0 */
> enum Alignment {
> Left    = 0,
> Right   = 1,
> Center  = 2,
> Unknown = 3,
> } __attribute__((__packed__));
> 
> ⬢ [acme@...lbox pahole]$
> 
> I'll fixup those two:
> 
> die__create_new_enumeration: DW_TAG_subprogram (0x2e) @ <0x4cb784b> not handled in a rust CU!
> tag__recode_dwarf_type: couldn't find name for function 0x4cd72bf, abstract_origin=0, specification=0x4cb784
> 
> I.e. support functions inside enumerations.
> 
> And sure this will be refused by the kernel, lots of stuff that have
> invalid names, probably need to turn those into void as well as a
> continuation of this hack, then prune, maybe that is it, we'll see.

Rather than voiding the names you can do something like this [0] to
coerce them into a format that the kernel is happy with. We initally
voided names but the resulting BTF was unusable since you couldn’t
lookup types by name.
 
A longer term fix is to relax the constraints in `__btf_name_char_ok`
which currently only allows characters that are valid in C
identifiers or part of a well-known section name like “.rodata”.

Even with relaxed constraints the kernel may still reject the BTF.
One example is the BTF PTR type since the kernel enforces that
name_offset = 0. When generating BTF for Rust code from LLVM IR
(and I assume from DWARF too) the name of PTR types is something
like `*const` or `*mut`.

- Dave

0: https://github.com/aya-rs/bpf-linker/blob/e4a9267b0fee69ecb2550058d3c8e5233f946ebe/src/llvm/di.rs#L34-L59


> Going AFK now.
> 
> - Arnaldo

Powered by blists - more mailing lists