Message-ID: <b586e946c8514cecde65f98de8e19eb276c09703.camel@HansenPartnership.com>
Date: Fri, 02 May 2025 09:30:53 -0400
From: James Bottomley <James.Bottomley@...senPartnership.com>
To: Thomas Weißschuh <linux@...ssschuh.net>
Cc: Masahiro Yamada <masahiroy@...nel.org>, Nathan Chancellor
 <nathan@...nel.org>, Arnd Bergmann <arnd@...db.de>, Luis Chamberlain
 <mcgrof@...nel.org>, Petr Pavlu <petr.pavlu@...e.com>, Sami Tolvanen
 <samitolvanen@...gle.com>, Daniel Gomez <da.gomez@...sung.com>, Paul Moore
 <paul@...l-moore.com>, James Morris <jmorris@...ei.org>, "Serge E. Hallyn"
 <serge@...lyn.com>, Jonathan Corbet <corbet@....net>, Madhavan Srinivasan
 <maddy@...ux.ibm.com>, Michael Ellerman <mpe@...erman.id.au>, Nicholas
 Piggin <npiggin@...il.com>, Christophe Leroy <christophe.leroy@...roup.eu>,
 Naveen N Rao <naveen@...nel.org>, Mimi Zohar <zohar@...ux.ibm.com>, Roberto
 Sassu <roberto.sassu@...wei.com>, Dmitry Kasatkin
 <dmitry.kasatkin@...il.com>,  Eric Snowberg <eric.snowberg@...cle.com>,
 Nicolas Schier <nicolas.schier@...ux.dev>, Fabian
 Grünbichler <f.gruenbichler@...xmox.com>, Arnout Engelen
 <arnout@...t.net>, Mattia Rizzolo <mattia@...reri.org>, kpcyrd
 <kpcyrd@...hlinux.org>, Christian Heusel <christian@...sel.eu>,
 Câju Mihai-Drosi <mcaju95@...il.com>, 
 linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org, 
 linux-arch@...r.kernel.org, linux-modules@...r.kernel.org, 
 linux-security-module@...r.kernel.org, linux-doc@...r.kernel.org, 
 linuxppc-dev@...ts.ozlabs.org, linux-integrity@...r.kernel.org
Subject: Re: [PATCH v3 0/9] module: Introduce hash-based integrity checking

On Fri, 2025-05-02 at 08:53 +0200, Thomas Weißschuh wrote:
> Hi James,
> 
> On 2025-04-29 10:05:04-0400, James Bottomley wrote:
> > On Tue, 2025-04-29 at 15:04 +0200, Thomas Weißschuh wrote:
> > > The current signature-based module integrity checking has some
> > > drawbacks in combination with reproducible builds:
> > > Either the module signing key is generated at build time, which
> > > makes the build unreproducible,
> > 
> > I don't believe it does: as long as you know what the key was,
> > which you can get from the kernel keyring, you can exactly
> > reproduce the core build (it's a public key after all and really
> > equivalent to built in configuration).  Is the fact that you have
> > to boot the kernel to get the key the problem?  In which case we
> > could insist it be shipped in the kernel packaging.
> 
> See below.
> 
> > >  or a static key is used, which precludes rebuilds by third
> > > parties and makes the whole build and packaging process much more
> > > complicated. 
> > 
> > No, it's the same as above ... as long as you have the public key
> > you can reproduce the core build with the same end to end hash.
> 
> While the scheme you propose does allow verification of
> rebuildability, it does not satisfy the requirements for a
> reproducible build as understood by the general reproducible builds
> community:
> 
> 	When is a build reproducible?
> 
> 	A build is reproducible if given the same source code, build
> environment
> 	and build instructions, any party can recreate bit-by-bit
> identical
> 	copies of all specified artifacts.
> 
> 	The relevant attributes of the build environment, the build
> instructions
> 	and the source code as well as the expected reproducible
> artifacts are
> 	defined by the authors or distributors. The artifacts of a
> build are the
> 	parts of the build results that are the desired primary
> output.
> 
> https://reproducible-builds.org/docs/definition/

That's a doctor doctor problem, and also if we adopted your
interpretation would mean that most kernel packages can never be
reproducible.  The clue is even in the "any party can recreate bit-by-
bit identical copies of all specified artifacts".

Under a your interpretation of the above, any signed binary isn't
"reproducible" even if the underlying build was, which means any secure
boot kernel would never be reproducible because it also has to be a
signed binary.  The solution is simple: can you strip the signature and
reproduce the build?  If yes, then the build is reproducible and even
fits with the "any party can recreate ..." above.   This is the
interpretation pretty much everyone else has been using.  It's why
people like Intel with source only availability and Intel build only
signing tout reproduceability: they only issue signed confidential VM
firmware, but you can technically reproduce the build of the firmware
minus the signature but you can never sign it.

> Specifically the output of any party can recreate bit-by-bit
> identical copies of all specified artifacta previous build (the
> public key, module signatures) is not available during the rebuild or
> verification.

You just strip the signatures before verifying reproducibility.

> > However, is there also a corresponding question of how we verify
> > reproduceability of kernel builds (and the associated modules ... I
> > assume for the modules you do strip the appended signature)?
> 
> Currently distros either don't enforce the reproducibility of the
> kernel package at all or disable MODULE_SIG.
> With the proposed scheme there would be no signatures on builtin
> modules.

If you take off the appended signature off the module, you can verify
reproduceability.

> > I assume you're going by the secure boot hash (authenticode hash of
> > the efi stub and the compressed payload which includes the key). 
> > However, if we had the vmlinux.o we could do a much more nuanced
> > hash to verify the build, say by placing the keyring data in a
> > section that isn't hashed.
> 
> The currently existing tooling does not have any nuance in its
> verifications. It just compares bit-by-bit.

So I think fixing that might be what you want.  All current secure
build processes (hermetic builds, SLSA and the like) are requiring
output provenance (i.e. signed artifacts).  If you try to stand like
Canute against this tide saying "no signed builds", you're simply
opposing progress for the sake of it, particularly when stripping the
signatures for comparison is easy.

> I think this is intentional as any bespoke per-package logic would
> introduce possible failure modes and stand in the way of implementing
> multiple completely independent verification toolsets.
> While bespoke tools like diffoscope exist, these are only for
> development and debugging. Not not for the reproducibiliy check
> itself.

So you think stripping signatures is failure prone?  If that were the
case then so would be verifying signatures upon which our whole secure
boot and signed module loading is based.


> How to handle secure-boot with distro keys is not yet clearly
> defined. I see two possibilities, which should be possible with the
> proposed scheme, both starting with the build of an unsigned kernel
> package. Then a signature would be computed on private infrastructure
> and either
> * shipped in a standalone package, to be combined with the kernel
> when that is installed to the ESP/flash etc.
> * used as input of a signed kernel package where it is combined with
> the unsigned kernel image.

Or you simply ship tools to remove the signature;

sbattach --remove <signed efi variable>

already does this for you ...

Regards,

James

Powered by blists - more mailing lists