Message-ID: <CA+icZUX_actkO4Si2HNQiH6AcS=3aNXA1OOka7LZNtEBQCNTGA@mail.gmail.com>
Date: Sat, 14 Dec 2024 14:04:19 +0100
From: Sedat Dilek <sedat.dilek@...il.com>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: Sami Tolvanen <samitolvanen@...gle.com>, Luis Chamberlain <mcgrof@...nel.org>, 
	Miguel Ojeda <ojeda@...nel.org>, Greg Kroah-Hartman <gregkh@...uxfoundation.org>, 
	Matthew Maurer <mmaurer@...gle.com>, Alex Gaynor <alex.gaynor@...il.com>, Gary Guo <gary@...yguo.net>, 
	Petr Pavlu <petr.pavlu@...e.com>, Daniel Gomez <da.gomez@...sung.com>, Neal Gompa <neal@...pa.dev>, 
	Hector Martin <marcan@...can.st>, Janne Grunau <j@...nau.net>, Miroslav Benes <mbenes@...e.cz>, 
	Asahi Linux <asahi@...ts.linux.dev>, linux-kbuild@...r.kernel.org, 
	linux-kernel@...r.kernel.org, linux-modules@...r.kernel.org, 
	rust-for-linux@...r.kernel.org
Subject: Re: [PATCH v6 18/18] Documentation/kbuild: Add DWARF module versioning

On Sat, Dec 14, 2024 at 12:34 PM Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> On Fri, Nov 22, 2024 at 5:43 AM Sami Tolvanen <samitolvanen@...gle.com> wrote:
> >
> > Add documentation for gendwarfksyms changes, and the kABI stability
> > features that can be useful for distributions even though they're not
> > used in mainline kernels.
> >
> > Signed-off-by: Sami Tolvanen <samitolvanen@...gle.com>
> > ---
> >  Documentation/kbuild/gendwarfksyms.rst | 308 +++++++++++++++++++++++++
> >  Documentation/kbuild/index.rst         |   1 +
> >  2 files changed, 309 insertions(+)
> >  create mode 100644 Documentation/kbuild/gendwarfksyms.rst
> >
> > diff --git a/Documentation/kbuild/gendwarfksyms.rst b/Documentation/kbuild/gendwarfksyms.rst
> > new file mode 100644
> > index 000000000000..7725d7f57131
> > --- /dev/null
> > +++ b/Documentation/kbuild/gendwarfksyms.rst
> > @@ -0,0 +1,308 @@
> > +=======================
> > +DWARF module versioning
> > +=======================
> > +
> > +1. Introduction
> > +===============
> > +
> > +When CONFIG_MODVERSIONS is enabled, symbol versions for modules
> > +are typically calculated from preprocessed source code using the
> > +**genksyms** tool.  However, this is incompatible with languages such
> > +as Rust, where the source code has insufficient information about
> > +the resulting ABI. With CONFIG_GENDWARFKSYMS (and CONFIG_DEBUG_INFO)
> > +selected, **gendwarfksyms** is used instead to calculate symbol versions
> > +from the DWARF debugging information, which contains the necessary
> > +details about the final module ABI.
> > +
> > +1.1. Usage
> > +==========
> > +
> > +gendwarfksyms accepts a list of object files on the command line, and a
> > +list of symbol names (one per line) in standard input::
> > +
> > +        Usage: gendwarfksyms [options] elf-object-file ... < symbol-list
> > +
> > +        Options:
> > +          -d, --debug          Print debugging information
> > +              --dump-dies      Dump DWARF DIE contents
> > +              --dump-die-map   Print debugging information about die_map changes
> > +              --dump-types     Dump type strings
> > +              --dump-versions  Dump expanded type strings used for symbol versions
> > +          -s, --stable         Support kABI stability features
> > +          -T, --symtypes file  Write a symtypes file
> > +          -h, --help           Print this message
> > +
> > +
> > +2. Type information availability
> > +================================
> > +
> > +While symbols are typically exported in the same translation unit (TU)
> > +where they're defined, it's also perfectly fine for a TU to export
> > +external symbols. For example, this is done when calculating symbol
> > +versions for exports in stand-alone assembly code.
> > +
> > +To ensure the compiler emits the necessary DWARF type information in the
> > +TU where symbols are actually exported, gendwarfksyms adds a pointer
> > +to exported symbols in the `EXPORT_SYMBOL()` macro using the following
> > +macro::
> > +
> > +        #define __GENDWARFKSYMS_EXPORT(sym)                             \
> > +                static typeof(sym) *__gendwarfksyms_ptr_##sym __used    \
> > +                        __section(".discard.gendwarfksyms") = &sym;
> > +
> > +
> > +When a symbol pointer is found in DWARF, gendwarfksyms can use its
> > +type for calculating symbol versions even if the symbol is defined
> > +elsewhere. The name of the symbol pointer is expected to start with
> > +`__gendwarfksyms_ptr_`, followed by the name of the exported symbol.
>
> I am interested in this sentence.
>
> __GENDWARFKSYMS_EXPORT() is primarily introduced to handle
> EXPORT_SYMBOL() in *.S files.
> In fact, .discard.gendwarfksyms is always output for EXPORT_SYMBOL()
> from *.c files.
>
> Can we make it faster by processing only symbol references in the
> .dscard.gendwarfksyms section, and skipping all other sections entirely?
>

Hi Masahiro,

I am interested in everything making my build faster.

v6 introduced a longer build-time for me.
Sami wanted to check after his vacancies see [1].

Just for the records.

Best regards,
-Sedat-

[1] https://lore.kernel.org/all/CABCJKudwpKX1_j46Tp6=eAJ0JU2zWE15+c8OFq9LMAnSqwesOw@mail.gmail.com/

>
>
> > +4.3. Adding structure members
> > +=============================
> > +
> > +Perhaps the most common ABI compatible changeis adding a member to a
>
> changeis -> change is
>
>
>
>
>
>
>
> > +kernel data structure. When changes to a structure are anticipated,
> > +distribution maintainers can pre-emptively reserve space in the
> > +structure and take it into use later without breaking the ABI. If
> > +changes are needed to data structures without reserved space, existing
> > +alignment holes can potentially be used instead. While kABI rules could
> > +be added for these type of changes, using unions is typically a more
> > +natural method. This section describes gendwarfksyms support for using
> > +reserved space in data structures and hiding members that don't change
> > +the ABI when calculating symbol versions.
> > +
> > +4.3.1. Reserving space and replacing members
> > +============================================
> > +
> > +Space is typically reserved for later use by appending integer types, or
> > +arrays, to the end of the data structure, but any type can be used. Each
> > +reserved member needs a unique name, but as the actual purpose is usually
> > +not known at the time the space is reserved, for convenience, names that
> > +start with `__kabi_` are left out when calculating symbol versions::
> > +
> > +        struct s {
> > +                long a;
> > +                long __kabi_reserved_0; /* reserved for future use */
> > +        };
> > +
> > +The reserved space can be taken into use by wrapping the member in a
> > +union, which includes the original type and the replacement member::
> > +
> > +        struct s {
> > +                long a;
> > +                union {
> > +                        long __kabi_reserved_0; /* original type */
> > +                        struct b b; /* replaced field */
> > +                };
> > +        };
> > +
> > +If the `__kabi_` naming scheme was used when reserving space, the name
> > +of the first member of the union must start with `__kabi_reserved`. This
> > +ensures the original type is used when calculating versions, but the name
> > +is again left out. The rest of the union is ignored.
> > +
> > +If we're replacing a member that doesn't follow this naming convention,
> > +we also need to preserve the original name to avoid changing versions,
> > +which we can do by changing the first union member's name to start with
> > +`__kabi_renamed` followed by the original name.
> > +
> > +The examples include `KABI_(RESERVE|USE|REPLACE)*` macros that help
> > +simplify the process and also ensure the replacement member is correctly
> > +aligned and its size won't exceed the reserved space.
> > +
> > +4.3.2. Hiding members
> > +=====================
> > +
> > +Predicting which structures will require changes during the support
> > +timeframe isn't always possible, in which case one might have to resort
> > +to placing new members into existing alignment holes::
> > +
> > +        struct s {
> > +                int a;
> > +                /* a 4-byte alignment hole */
> > +                unsigned long b;
> > +        };
> > +
> > +
> > +While this won't change the size of the data structure, one needs to
> > +be able to hide the added members from symbol versioning. Similarly
> > +to reserved fields, this can be accomplished by wrapping the added
> > +member to a union where one of the fields has a name starting with
> > +`__kabi_ignored`::
> > +
> > +        struct s {
> > +                int a;
> > +                union {
> > +                        char __kabi_ignored_0;
> > +                        int n;
> > +                };
> > +                unsigned long b;
> > +        };
> > +
> > +With **--stable**, both versions produce the same symbol version.
> > diff --git a/Documentation/kbuild/index.rst b/Documentation/kbuild/index.rst
> > index cee2f99f734b..e82af05cd652 100644
> > --- a/Documentation/kbuild/index.rst
> > +++ b/Documentation/kbuild/index.rst
> > @@ -21,6 +21,7 @@ Kernel Build System
> >      reproducible-builds
> >      gcc-plugins
> >      llvm
> > +    gendwarfksyms
> >
> >  .. only::  subproject and html
> >
> > --
> > 2.47.0.371.ga323438b13-goog
> >
>
>
> --
> Best Regards
>
>
>
> Masahiro Yamada

Powered by blists - more mailing lists