linux-kernel - Re: [PATCH v2 4/7] scripts: generate_rust

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAJ-ks9nNn753cJrhQkiRA41gXhsFMz5_gWzdGgAqKiGOkUbaDQ@mail.gmail.com>
Date: Wed, 19 Mar 2025 18:25:25 -0400
From: Tamir Duberstein <tamird@...il.com>
To: Trevor Gross <tmgross@...ch.edu>
Cc: Miguel Ojeda <ojeda@...nel.org>, Alex Gaynor <alex.gaynor@...il.com>, 
	Boqun Feng <boqun.feng@...il.com>, Gary Guo <gary@...yguo.net>, 
	Björn Roy Baron <bjorn3_gh@...tonmail.com>, 
	Benno Lossin <benno.lossin@...ton.me>, Andreas Hindborg <a.hindborg@...nel.org>, 
	Alice Ryhl <aliceryhl@...gle.com>, Danilo Krummrich <dakr@...nel.org>, 
	Boris-Chengbiao Zhou <bobo1239@....de>, Kees Cook <kees@...nel.org>, Fiona Behrens <me@...enk.dev>, 
	rust-for-linux@...r.kernel.org, linux-kernel@...r.kernel.org, 
	Lukas Wirth <lukas.wirth@...rous-systems.com>
Subject: Re: [PATCH v2 4/7] scripts: generate_rust_analyzer.py: add type hints

On Wed, Mar 19, 2025 at 5:25 PM Trevor Gross <tmgross@...ch.edu> wrote:
>
> On Tue, Mar 11, 2025 at 9:18 PM Tamir Duberstein <tamird@...il.com> wrote:
> >
> > Python type hints allow static analysis tools like mypy to detect type
> > errors during development, improving the developer experience.
> >
> > Python type hints have been present in the kernel since 2019 at the
> > latest; see commit 6ebf5866f2e8 ("kunit: tool: add Python wrappers for
> > running KUnit tests").
> >
> > Run `uv tool run mypy --strict scripts/generate_rust_analyzer.py` to
> > verify.
>
> From the discussion, it may be better to instead mention the direct
> invocation (without uv).

👍 done.

> Could you also mention the target min version? Since apparently the
> kernel has a spread. It looks like maybe 3.8 based on what is used
> here.

Added, it is now 3.8 because mypy doesn't support anything lower.

>
> > This removes `"is_proc_macro": false` from `rust-project.json` in
> > exchange for stricter types. This field is interpreted as false if
> > absent[1] so this doesn't change the behavior of rust-analyzer.
> >
> > Link: https://github.com/rust-lang/rust-analyzer/blob/8d01570b5e812a49daa1f08404269f6ea5dd73a1/crates/project-model/src/project_json.rs#L372-L373 [1]
> > Signed-off-by: Tamir Duberstein <tamird@...il.com>
> > ---
> >  scripts/generate_rust_analyzer.py | 130 ++++++++++++++++++++++++++++----------
> >  1 file changed, 96 insertions(+), 34 deletions(-)
> >
> > diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> > index 7e78b926e61f..c73ea8d116a4 100755
> > --- a/scripts/generate_rust_analyzer.py
> > +++ b/scripts/generate_rust_analyzer.py
> > @@ -10,8 +10,10 @@ import os
> >  import pathlib
> >  import subprocess
> >  import sys
> > +import typing as T
>
> Nit: is there any need to keep everything namespaced? I think it
> should be fine to import `Iterable` `TypedDict` etc directly since
> they aren't confusable.

I saw this style in another project and liked it, but I don't feel
strongly. Changed.

>
> Same for `pathlib.Path` since there is no other `Path` (some of that
> is preexisting).

I'll leave this one as is to avoid more churn.

>
> > +    def append_proc_macro_crate(
> > +        display_name: str,
> > +        root_module: pathlib.Path,
> > +        deps: list[str],
> > +        cfg: list[str] = [],
> > +    ) -> None:
> > +        append_crate(display_name, root_module, deps, cfg)
> > +        proc_macro_dylib_name = subprocess.check_output(
> > +            [os.environ["RUSTC"], "--print", "file-names", "--crate-name", display_name, "--crate-type", "proc-macro", "-"],
>
> Nit, may as well use this opportunity to wrap the line.

I considered that, but when you use git with `-w` this line doesn't
change, so I'd rather leave it.

>
> > +            stdin=subprocess.DEVNULL,
> > +        ).decode('utf-8').strip()
> > +        crate: ProcMacroCrate = {
> > +            **crates[-1],
> > +            "is_proc_macro": True,
> > +            "proc_macro_dylib_path": f"{objtree}/rust/{proc_macro_dylib_name}",
> >          }
> > -        if is_proc_macro:
> > -            proc_macro_dylib_name = subprocess.check_output(
> > -                [os.environ["RUSTC"], "--print", "file-names", "--crate-name", display_name, "--crate-type", "proc-macro", "-"],
> > -                stdin=subprocess.DEVNULL,
> > -            ).decode('utf-8').strip()
> > -            crate["proc_macro_dylib_path"] = f"{objtree}/rust/{proc_macro_dylib_name}"
> > -        crates_indexes[display_name] = len(crates)
> > -        crates.append(crate)
> > +        crates[-1] = crate
>
> The unpacking is a bit confusing here, can `crates[-1]` just be set
> rather than duplicating and replacing it?
>
> Maybe the body of `append_crate` should be `build_crate(...) -> Crate`
> (which could then be a top-level function), then `append_crate`,
> `append_crate_with_generated`, etc call that and handle modification /
> appending themselves.

This is a nice improvement. Done.

> > +        crate: CrateWithGenerated = {
> > +            **crates[-1],
> > +            "source": {
> > +                "include_dirs": [
> > +                    str(srctree / "rust" / display_name),
> > +                    str(objtree / "rust")
> > +                ],
> > +                "exclude_dirs": [],
> > +            }
> >          }
> > +        crates[-1] = crate

👍

>
> - Trevor

Thanks for the review!