[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20241030170106.1501763-21-samitolvanen@google.com>
Date: Wed, 30 Oct 2024 17:01:07 +0000
From: Sami Tolvanen <samitolvanen@...gle.com>
To: Masahiro Yamada <masahiroy@...nel.org>, Luis Chamberlain <mcgrof@...nel.org>,
Miguel Ojeda <ojeda@...nel.org>, Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc: Matthew Maurer <mmaurer@...gle.com>, Alex Gaynor <alex.gaynor@...il.com>,
Gary Guo <gary@...yguo.net>, Petr Pavlu <petr.pavlu@...e.com>,
Daniel Gomez <da.gomez@...sung.com>, Neal Gompa <neal@...pa.dev>, Hector Martin <marcan@...can.st>,
Janne Grunau <j@...nau.net>, Miroslav Benes <mbenes@...e.cz>, Asahi Linux <asahi@...ts.linux.dev>,
Sedat Dilek <sedat.dilek@...il.com>, linux-kbuild@...r.kernel.org,
linux-kernel@...r.kernel.org, linux-modules@...r.kernel.org,
rust-for-linux@...r.kernel.org, Sami Tolvanen <samitolvanen@...gle.com>
Subject: [PATCH v5 00/19] Implement DWARF modversions
Hi,
Here's v5 of the DWARF modversions series. The main motivation is
modversions support for Rust, which is important for distributions
like Android that are about to ship Rust kernel modules. Per Luis'
request [1], v2 dropped the Rust specific bits from the series and
instead added the feature as an option for the entire kernel to
make it easier to evaluate the benefits of this approach, and to
get better test coverage. Matt is addressing Rust modversion_info
compatibility issues in a separate patch set [2] that depends on this
series, and actually allows modversions to be enabled with Rust.
Short background: Unlike C, Rust source code doesn't have sufficient
information about the final ABI, as the compiler has considerable
freedom in adjusting structure layout, for example, which makes
using a source code parser like genksyms a non-starter. Based on
earlier feedback, this series uses DWARF debugging information for
computing versions. DWARF is an established and a relatively stable
format, which includes all the necessary ABI details, and adding a
CONFIG_DEBUG_INFO dependency for Rust symbol versioning seems like a
reasonable trade-off as most distributions already enable it.
The first patch moves the genksyms CRC32 implementation to a shared
header file to avoid code duplication and the next 15 patches add
gendwarfksyms, a tool for computing symbol versions from DWARF. When
passed a list of exported symbols and object files, the tool
generates an expanded type string for each symbol and computes symbol
CRCs similarly to genksyms. gendwarfksyms is written in C and uses
libdw to process DWARF. Patch 17 ensures that debugging information
is present where we need it, patch 18 adds gendwarfksyms as an
alternative to genksyms, and the last patch adds documentation.
v5 is based on v6.12-rc5 and for your convenience the series is also
available here:
https://github.com/samitolvanen/linux/commits/gendwarfksyms-v5
If you also want to test the series with Rust modules, this branch
adds Matt's latest modversion_info series:
https://github.com/samitolvanen/linux/commits/rustmodversions-v5
Sami
[1] https://lore.kernel.org/lkml/ZnIZEtkkQWEIGf9n@bombadil.infradead.org/
[2] https://lore.kernel.org/lkml/20240925233854.90072-1-mmaurer@google.com/
---
v5:
- Rebased on v6.12-rc5.
- Fixed an issue with limiting structure expansion, and applied
Petr's clean-up. (Patch 10)
- Dropped an unnecessary return statement in error path. (Patch
12)
- Addressed several other smaller issues Petr brought up. (Patches
13, 14, and 15)
- Added a KBUILD_GENDWARFKSYMS_STABLE flag to enable --stable for
the entire kernel build. (Patch 18)
- Updated documentation to include KBUILD flags. (Patch 19)
- Picked up Reviewed-by tags from v4.
v4: https://lore.kernel.org/lkml/20241008183823.36676-21-samitolvanen@google.com/
- Rebased on v6.12-rc2, which now includes all the prerequisites.
- Dropped unnecessary name_only parameter for symbols.c::for_each
and cleaned up error handling. (Patch 3)
- Fixed anonymous scope handling to ensure unnamed DIEs don't get
names. (Patch 4)
- Added non-variant children to variant_type output, and included
DW_AT_discr_value attributes for variants. (Patch 9)
- Added another symbol pointer test case. (Patch 16)
- Picked up (Acked|Reviewed)-by tags from v3.
v3: https://lore.kernel.org/lkml/20240923181846.549877-22-samitolvanen@google.com/
- Updated SPX license headers.
- Squashed the first two patches in v2 and tried to reduce churn as
much as reasonable.
- Dropped patch 18 from v2 ("x86/asm-prototypes: Include
<asm/ptrace.h>") as it's addressed by a separate patch.
- Changed the error handling code to immediately terminate instead
of propagating the errors back to main, which cleaned up the code
quite a bit.
- Switched to the list and hashtable implementations in scripts and
dropped the remaining tools/include dependencies. Added a couple
missing list macros. (patch 1)
- Moved the genksyms CRC32 implementation to scripts/include and
dropped the duplicate code. (patches 2 and 14)
- Switched from ad-hoc command line parsing to getopt_long (patch 3).
- Added structure member and function parameter names to the DIE
output to match genksyms behavior, and tweaked the symtypes format
to be more parser-friendly in general based on Petr's suggestions.
- Replaced the declaration-only struct annotations with more generic
kABI stability rules that allow source code annotations to be used
where #ifndef __GENKSYMS__ was previously used. Added support for
rules that can be used to exclude enumerators from versioning.
(patch 16)
- Per Miroslav's suggestion, added an option to hide structure
members from versioning when they're added to existing alignment
holes, for example. (patch 16)
- Per Greg's request, added documentation and example macros for the
--stable features, and a couple of test cases. (patches 15, 16, and
20)
- Fixed making symtypes files, which need to depend on .o files with
gendwarfksyms. (patch 19)
- Addressed several other smaller issues that Petr and Masahiro
kindly pointed out during the v2 review.
v2: https://lore.kernel.org/lkml/20240815173903.4172139-21-samitolvanen@google.com/
- Per Luis' request, dropped Rust-specific patches and added
gendwarfksyms as an alternative to genksyms for the entire
kernel.
- Added support for missing DWARF features needed to handle
also non-Rust code.
- Changed symbol address matching to use the symbol table
information instead of relying on addresses in DWARF.
- Added __gendwarfksyms_ptr patches to ensure the compiler emits
the necessary type information in DWARF even for symbols that
are defined in other TUs.
- Refactored debugging output and moved the more verbose output
behind --dump* flags.
- Added a --symtypes flag for generating a genksyms-style
symtypes output based on Petr's feedback, and refactored
symbol version calculations to be based on symtypes instead
of raw --dump-dies output.
- Based on feedback from Greg and Petr, added --stable flag and
support for reserved data structure fields and declaration-onl
structures. Also added examples for using these features.
- Added a GENDWARFKSYMS option and hooked up kbuild support
for both C and assembly code. Note that with gendwarfksyms,
we have to actually build a temporary .o file for calculating
assembly modversions.
v1: https://lore.kernel.org/lkml/20240617175818.58219-17-samitolvanen@google.com/
---
Sami Tolvanen (19):
scripts: move genksyms crc32 implementation to a common include
tools: Add gendwarfksyms
gendwarfksyms: Add address matching
gendwarfksyms: Expand base_type
gendwarfksyms: Add a cache for processed DIEs
gendwarfksyms: Expand type modifiers and typedefs
gendwarfksyms: Expand subroutine_type
gendwarfksyms: Expand array_type
gendwarfksyms: Expand structure types
gendwarfksyms: Limit structure expansion
gendwarfksyms: Add die_map debugging
gendwarfksyms: Add symtypes output
gendwarfksyms: Add symbol versioning
gendwarfksyms: Add support for kABI rules
gendwarfksyms: Add support for reserved and ignored fields
gendwarfksyms: Add support for symbol type pointers
export: Add __gendwarfksyms_ptr_ references to exported symbols
kbuild: Add gendwarfksyms as an alternative to genksyms
Documentation/kbuild: Add DWARF module versioning
Documentation/kbuild/gendwarfksyms.rst | 276 +++++
Documentation/kbuild/index.rst | 1 +
include/linux/export.h | 15 +
kernel/module/Kconfig | 31 +
scripts/Makefile | 3 +-
scripts/Makefile.build | 41 +-
scripts/gendwarfksyms/.gitignore | 2 +
scripts/gendwarfksyms/Makefile | 12 +
scripts/gendwarfksyms/cache.c | 44 +
scripts/gendwarfksyms/die.c | 166 +++
scripts/gendwarfksyms/dwarf.c | 1109 +++++++++++++++++++
scripts/gendwarfksyms/examples/kabi.h | 141 +++
scripts/gendwarfksyms/examples/kabi_ex0.c | 86 ++
scripts/gendwarfksyms/examples/kabi_ex1.c | 89 ++
scripts/gendwarfksyms/examples/kabi_ex2.c | 98 ++
scripts/gendwarfksyms/examples/kabi_rules.c | 56 +
scripts/gendwarfksyms/examples/symbolptr.c | 33 +
scripts/gendwarfksyms/gendwarfksyms.c | 185 ++++
scripts/gendwarfksyms/gendwarfksyms.h | 288 +++++
scripts/gendwarfksyms/kabi.c | 263 +++++
scripts/gendwarfksyms/symbols.c | 339 ++++++
scripts/gendwarfksyms/types.c | 477 ++++++++
scripts/genksyms/genksyms.c | 77 +-
scripts/include/crc32.h | 93 ++
24 files changed, 3840 insertions(+), 85 deletions(-)
create mode 100644 Documentation/kbuild/gendwarfksyms.rst
create mode 100644 scripts/gendwarfksyms/.gitignore
create mode 100644 scripts/gendwarfksyms/Makefile
create mode 100644 scripts/gendwarfksyms/cache.c
create mode 100644 scripts/gendwarfksyms/die.c
create mode 100644 scripts/gendwarfksyms/dwarf.c
create mode 100644 scripts/gendwarfksyms/examples/kabi.h
create mode 100644 scripts/gendwarfksyms/examples/kabi_ex0.c
create mode 100644 scripts/gendwarfksyms/examples/kabi_ex1.c
create mode 100644 scripts/gendwarfksyms/examples/kabi_ex2.c
create mode 100644 scripts/gendwarfksyms/examples/kabi_rules.c
create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c
create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c
create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h
create mode 100644 scripts/gendwarfksyms/kabi.c
create mode 100644 scripts/gendwarfksyms/symbols.c
create mode 100644 scripts/gendwarfksyms/types.c
create mode 100644 scripts/include/crc32.h
base-commit: 81983758430957d9a5cb3333fe324fd70cf63e7e
--
2.47.0.163.g1226f6d8fa-goog
Powered by blists - more mailing lists