linux-kernel - Re: [PATCH] rust: bindgen: upgrade to 0.65.1

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Tue, 15 Aug 2023 00:43:02 +0200
From:   Miguel Ojeda <miguel.ojeda.sandonis@...il.com>
To:     Aakash Sen Sharma <aakashsensharma@...il.com>
Cc:     corbet@....net, ojeda@...nel.org, alex.gaynor@...il.com,
        wedsonaf@...il.com, boqun.feng@...il.com, gary@...yguo.net,
        bjorn3_gh@...tonmail.com, nathan@...nel.org,
        ndesaulniers@...gle.com, trix@...hat.com, masahiroy@...nel.org,
        me@...enk.de, aliceryhl@...gle.com, benno.lossin@...ton.me,
        dev@...lasmohrin.de, lina@...hilina.net, hca@...ux.ibm.com,
        rust-for-linux@...r.kernel.org, linux-doc@...r.kernel.org,
        linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH] rust: bindgen: upgrade to 0.65.1

On Mon, Jun 12, 2023 at 9:45 PM Aakash Sen Sharma
<aakashsensharma@...il.com> wrote:
>
> * Rationale:
>
> Upgrades bindgen to code-generation for anonymous unions, structs, and enums [7]
> for LLVM-16 based toolchains.
>
> The following upgrade also incorporates `noreturn` support from bindgen
> allowing us to remove useless `loop` calls which was placed as a
> workaround.
>
> * Truncated build logs on using bindgen `v0.56.0` prior to LLVM-16 toolchain:
>
> ```
> $ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
>   RUSTC L rust/core.o
>   BINDGEN rust/bindings/bindings_generated.rs
>   BINDGEN rust/bindings/bindings_helpers_generated.rs
>   BINDGEN rust/uapi/uapi_generated.rs
> thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
> ...
> thread 'main' panicked at '"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)" is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
> ...
> ```
>
> * LLVM-16 Changes:
>
> API changes [1] were introduced such that libclang would emit names like
> "(unnamed union at compiler_types.h:146:2)" for unnamed unions, structs, and
> enums whereas it previously returned an empty string.
>
> * Bindgen Changes:
>
> Bindgen `v0.56.0` on LLVM-16 based toolchains hence was unable to process the
> anonymous union in `include/linux/compiler_types` `struct ftrace_branch_data`
> and caused subsequent panics as the new `libclang` API emitted name was not
> being handled. The following issue was fixed in Bindgen `v0.62.0` [2].
>
> Bindgen `v0.58.0` changed the flags `--whitelist-*` and `--blacklist-*`
> to `--allowlist-*` and `--blocklist-*` respectively [3].
>
> Bindgen `v0.61.0` added support for `_Noreturn`, `[[noreturn]]`, `__attribute__((noreturn))` [4],
> hence the empty `loop`s used to circumvent bindgen returning `!` in place of `()`
> for noreturn attributes have been removed completely.
>
> Bindgen `v0.61.0` also changed default functionality to bind `size_t` to `usize` [5] and
> added the `--no-size_t-is-usize` [5] flag to not bind `size_t` as `usize`.
>
> Bindgen `v0.65.0` removed `--size_t-is-usize` flag [6].
>
> Link: https://github.com/llvm/llvm-project/commit/19e984ef8f49bc3ccced15621989fa9703b2cd5b [1]
> Link: https://github.com/rust-lang/rust-bindgen/pull/2319 [2]
> Link: https://github.com/rust-lang/rust-bindgen/pull/1990 [3]
> Link: https://github.com/rust-lang/rust-bindgen/issues/2094 [4]
> Link: https://github.com/rust-lang/rust-bindgen/commit/cc78b6fdb6e829e5fb8fa1639f2182cb49333569 [5]
> Link: https://github.com/rust-lang/rust-bindgen/pull/2408 [6]
> Closes: https://github.com/Rust-for-Linux/linux/issues/1013 [7]
> Signed-off-by: Aakash Sen Sharma <aakashsensharma@...il.com>

Applied to `rust-next`, thanks everyone!

I also did these extra changes:

    [ Reworded commit message. Mentioned the `bindgen-cli` binary crate
      change, linked to it and updated the Quick Start guide. Re-added a
      deleted "as" word in a code comment and reflowed comment to respect
      the maximum length. ]

In particular, I have reworded the commit message a fair bit:

    In LLVM 16, anonymous items may return names like `(unnamed union at ..)`
    rather than empty names [1], which breaks Rust-enabled builds because
    bindgen assumed an empty name instead of detecting them via
    `clang_Cursor_isAnonymous` [2]:

        $ make rustdoc LLVM=1 CLIPPY=1 -j$(nproc)
          RUSTC L rust/core.o
          BINDGEN rust/bindings/bindings_generated.rs
          BINDGEN rust/bindings/bindings_helpers_generated.rs
          BINDGEN rust/uapi/uapi_generated.rs
        thread 'main' panicked at
'"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)"
is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
        ...
        thread 'main' panicked at
'"ftrace_branch_data_union_(anonymous_at__/_/include/linux/compiler_types_h_146_2)"
is not a valid Ident', .../proc-macro2-1.0.24/src/fallback.rs:693:9
        ...

    This was fixed in bindgen 0.62.0. Therefore, upgrade bindgen to
    a more recent version, 0.65.1, to support LLVM 16.

    Since bindgen 0.58.0 changed the `--{white,black}list-*` flags to
    `--{allow,block}list-*` [3], update them on our side too.

    In addition, bindgen 0.61.0 moved its CLI utility into a binary crate
    called `bindgen-cli` [4]. Thus update the installation command in the
    Quick Start guide.

    Moreover, bindgen 0.61.0 changed the default functionality to bind
    `size_t` to `usize` [5] and added the `--no-size_t-is-usize` flag
    to not bind `size_t` as `usize`. Then bindgen 0.65.0 removed
    the `--size_t-is-usize` flag [6]. Thus stop passing the flag to bindgen.

    Finally, bindgen 0.61.0 added support for the `noreturn` attribute (in
    its different forms) [7]. Thus remove the infinite loop in our Rust
    panic handler after calling `BUG()`, since bindgen now correctly
    generates a `BUG()` binding that returns `!` instead of `()`.

    Link: https://github.com/llvm/llvm-project/commit/19e984ef8f49bc3ccced15621989fa9703b2cd5b
[1]
    Link: https://github.com/rust-lang/rust-bindgen/pull/2319 [2]
    Link: https://github.com/rust-lang/rust-bindgen/pull/1990 [3]
    Link: https://github.com/rust-lang/rust-bindgen/pull/2284 [4]
    Link: https://github.com/rust-lang/rust-bindgen/commit/cc78b6fdb6e829e5fb8fa1639f2182cb49333569
[5]
    Link: https://github.com/rust-lang/rust-bindgen/pull/2408 [6]
    Link: https://github.com/rust-lang/rust-bindgen/issues/2094 [7]

Aakash: please let me know if that is not OK!

Cheers,
Miguel