Message-ID: <hzjr4s6bp35apkmeyqaafsbhfrnetfzsarw34o5c5fvnuppfmz@n4yy43fdkpwx>
Date: Fri, 21 Feb 2025 15:08:01 -0600
From: Lucas De Marchi <lucas.demarchi@...el.com>
To: <linux-modules@...r.kernel.org>, <linux-kernel@...r.kernel.org>
CC: Emil Velikov <emil.l.velikov@...il.com>, Tobias Stoeckmann
	<tobias@...eckmann.org>, Lucas De Marchi <lucas.de.marchi@...il.com>, "Martin
 Wilck" <martin_wilck@....de>, Martin Wilck <mwilck@...e.com>, Michal Suchanek
	<msuchanek@...e.de>, Christian Hesse <mail@...rm.de>, Enrico Joerns
	<ejo@...gutronix.de>, Grayson Nocera <gnocera@...due.edu>, Max Kunzelmann
	<maxdev@...teo.de>, Sedat Dilek <sedat.dilek@...il.com>, Luis Chamberlain
	<mcgrof@...nel.org>, Petr Pavlu <petr.pavlu@...e.com>, Sami Tolvanen
	<samitolvanen@...gle.com>, Daniel Gomez <da.gomez@...sung.com>
Subject: [ANNOUNCE] kmod 34

kmod 34 is out:

	https://www.kernel.org/pub/linux/utils/kernel/kmod/kmod-34.tar.xz
	https://www.kernel.org/pub/linux/utils/kernel/kmod/kmod-34.tar.sign

By number of commits this is the biggest release ever, although we
managed to do it without libkmod additions/removals. There were lots of
changes on improving tools, tests, CI, docs, bug fixes and changing the
build system.

The recommended build system for distros is now meson. Autotools is
still supported, but for this release only.

The shortlog is way too big, so I'm only adding the summary below:

    283  Emil Velikov <emil.l.velikov@...il.com>
    135  Tobias Stoeckmann <tobias@...eckmann.org>
     84  Lucas De Marchi <lucas.de.marchi@...il.com>
      4  Martin Wilck <martin_wilck@....de>
      4  Martin Wilck <mwilck@...e.com>
      2  Michal Suchanek <msuchanek@...e.de>
      2  dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
      1  Christian Hesse <mail@...rm.de>
      1  Enrico Joerns <ejo@...gutronix.de>
      1  Grayson Nocera <gnocera@...due.edu>
      1  Max Kunzelmann <maxdev@...teo.de>
      1  Sedat Dilek <sedat.dilek@...il.com>

Thanks a lot for everybody involved in this release. The complete NEWS
is below.

- Improvements

	- Drop pre-built .ko modules from git - distros/packages will need the
	  linux-headers to be able to run the testsuite. There was limited use
	  of the feature, while linters complained about "source-not-included"
	  or "source-contains-prebuilt-binary".

	- Switch build system to meson: autotools is still supported but slated
	  for removal on next release. This is the transition release to help
	  distros and integrators to move to the new build system. Default options
	  target distros while developers can use the build-dev.ini configuration
	  file.

	- Allow to load decompression libraries ondemand: liblzma.so, libz.so,
	  libxz.so and libzstd.so can now be loaded ondemand, only when there is
	  such a need. For use during early boot for loading modules, if
	  configured well it means none of these libraries are loaded: the
	  module loading logic via finit_module() will just hand over to kernel
	  the open file descriptor and kernel will use its own decompress routine.

	  If kernel doesn't handle decompression or if the module is compressed
	  with a different algorithm than the one configured in the kernel,
	  libkmod can still open the module by dynamically loading the
	  correspondent library.

	  Tools inspecting the module contents, like modinfo, will load that
	  single decompression library instead all of them.

	  For distros building with meson it's possible to choose the behavior
	  per library. Examples: a) -Ddlopen=all uses dlopen behavior for all
	  the libraries; b) -Ddlopen=xz, will make only xz to be dlopen'ed
	  while other enabled libraries will be linked in at build time.

	  The use of dlopen is annotated in the ELF file by using the ELF
	  Package Metadata spec as documented in
	  https://systemd.io/ELF_PACKAGE_METADATA/. Example:

	  $ dlopen-notes.py libkmod.so
	  # build/libkmod.so
	  [
	    {
	      "feature": "xz",
	      "description": "Support for uncompressing xz-compressed modules",
	      "priority": "recommended",
	      "soname": [
	        "liblzma.so.5"
	      ]
	    }
	  ]

	- Add -m / --moduledir to depmod to override in runtime the module
	  directory that was already possible to set on build time. Document
	  the interaction between the dir options: base, module and output.

	- Better error propagation in libkmod for its internal APIs and libc
	  functions up to the callers.

	- Improve libkmod API documentation by adding new sections, documenting
	  functions previously missing, rewording existing ones, adding version
	  information, cross-referencing, etc.

	- Remove deprecated arguments for depmod: --unresolved-error, --quiet,
	  --root and --map.

	- Remove deprecated arguments for rmmod: -w.

	- Remove deprecated arguments for insmod: -p and -s.

	- Add --syslog and --force for insmod to normalize it with other tools.

	- Add bash, fish and zsh shell-completion for insmod, rmmod and lsmod.

	- Remove depmod_module_directory_override from .pc as the kernel side
	  is not making use of it and will likely not need it.

	- Improve builtin module listing and retrieving information from its
	  modinfo index which reduces the amount of needed syscalls by 90%.

	- Improve zstd decompression by using streaming bufferless mode which
	  reduces the amount of syscalls by 65%.

	- Increase use of pread while parsing ELF and indexes in order to reduce
	  syscalls and improve performance.

	- Improve module sorting in depmod to speedup the use of the
	  modules.order index and support duplicate lines in it.

	- Avoid misaligned memory access while reading module signature in
	  libkmod.

	- Add more documentation for contributing to kmod. New developers are
	  welcome to look at the new README.md and CONTRIBUTING.md files for
	  information on process, coding style, build/installation, etc.

	- Overhaul man pages with multiple clarifications, section rewrites and
	  additional documentation.

	- Drop --with-rootlibdir as it's seldom used and was partially broken.

	- Drop strndupa() and alloca() for increased libc compatibility.

	- Better handling of LFS for increased compatibility with libc.

	- Protect kmod_get_dirname() and kmod_new() against NULL argument.

	- Normalize --version / --help output across all tools.

	- Always include log priority in messages, even when building with debug.

	- Optimize index reading by lazily reading nodes on demand, reducing
	  FILE overhead and reducing code duplication wrt FILE vs mmap
	  implementations, etc.

	- Switch index to pre-order to improve performance in both read and
	  write, meaning faster lookup and faster depmod. Some examples:
	  a) traversing all indexes via configuration dump shows a 9%
	  improvement on Raspberry Pi 2. b) writing the indexes takes 90% less
	  lseek() calls, leading to a performance gain of 13%.

	- Make symlink install locations more similar to what distros are
	  using: by default it installs the kmod binary as bin/kmod and the
	  symlinks are located in e.g. `sbin/depmod -> ../bin/kmod`. Changing
	  the sbin location is sufficient to move the symlinks to the
	  appropriate place, so distros using `--sbin /usr/bin` will have them
	  installed in that directory. This avoids distros having to remove the
	  symlink and add the symlinks by themselves. (meson only)

	- Install configuration directories,
	  /{etc,usr/lib}/{depmod,modprobe}.d/ as part of installation, matching
	  what several distros do during packaging. (mson only)

- Bug fixes

	- Fix testsuite using when using configurable module dir.

	- Fix typos on documentation and source code.

	- Fix out of bound access in multiple places when using long paths,
	  synthetic huge files, or handling memory allocation errors, or
	  inconsistent variable types, particularly on 32b builds.

	- Fix internal array APIs, with better error checking: improve execution on
	  very memory-constrained scenarios or very long paths.

	- Fix absolute path handling in depmod.

	- Fix libkmod memory leaks on error handling when getting builtin
	  module list.

	- Do not crash on invalid modules.builtin.modinfo file.

	- Fix link with lld resulting in empty testsuite.

	- Fix testsuite build/execution with musl.

- Others

	- Adopt clang-format and editorconfig for coding style and setup CI
	  action to keep the codebase consistent.

	- Adopt codespell in CI.

	- Adopt CodeQL integration in CI.

	- Adopt Codecov in CI.

	- Adopt SPDX copyright and license identifiers throughout the project.

	- Add more distros to CI, 32b builds, clang as compiler and lld as
	  linker.

	- Add support for clang sanitizers and squelch warnings.

	- Add tests for builtin modules from modinfo index file.

	- Multiple testsuite refactors and fixes to make it simpler to write tests.

	- Add CI coverage for docs

	- Improve strbuf implementation with more error checks and generalize
	  it to cover the role of scratchbuf. This allows to remove the
	  scratchbuf implementation.

	- Use common array and strbuf code in depmod to remove duplication.

	- Add abstraction and use more compiler builtins for addition and
	  multiplication with overflow checking.

	- Normalize use of C attributes throughout the project.


cheers,
Lucas De Marchi

Powered by blists - more mailing lists