lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <fe53ad80-6eeb-495a-a870-9c42b71f9887@gtucker.io>
Date: Tue, 14 Oct 2025 16:31:55 +0200
From: Guillaume Tucker <gtucker@...cker.io>
To: Miguel Ojeda <miguel.ojeda.sandonis@...il.com>
Cc: Nathan Chancellor <nathan@...nel.org>,
 Nicolas Schier <nicolas.schier@...ux.dev>, Miguel Ojeda <ojeda@...nel.org>,
 rust-for-linux@...r.kernel.org, linux-kbuild@...r.kernel.org,
 linux-kernel@...r.kernel.org, automated-testing@...ts.yoctoproject.org,
 Arnd Bergmann <arnd@...db.de>, workflows@...r.kernel.org,
 llvm@...ts.linux.dev
Subject: Re: [RFC PATCH 1/1] kbuild: add Makefile.container with CONTAINER
 option

Hi Miguel,

On 14/10/2025 1:58 pm, Miguel Ojeda wrote:
> On Tue, Oct 14, 2025 at 11:45 AM Guillaume Tucker <gtucker@...cker.io> wrote:
>>
>> Add scripts/Makefile.container to wrap the make command in a container
>> using the CONTAINER= variable to specify the image name.  For example:
>>
>>      make -f scripts/Makefile.container CONTAINER=korg-gcc defconfig
>>
>> The container image name is entirely arbitrary and the container tool
>> may be Docker, Podman or any other compatible alternative specified by
>> the CONTAINER_COMMAND variable.  The default is set to docker for now.
> 
> IIUC, this wraps reruns `make` inside the container, but it means
> hardcoding a particular tool and path, right? (unless one sets even
> more variables)

This is what the CONTAINER_COMMAND variable is for.  You can easily
set it in the environment e.g. `export CONTAINER_COMMAND=podman` or
anything and be done with it.  The default is set to `docker`.  As
far as I know, there's no need for anything else than the container
command and image name.  So it's not more hard-coded than say, the
default $(CC) executable.

> The cover letter says one can create an alias for this, but one could
> also do that for the underlying call anyway, unless I am missing
> something. And if we do this, then I would prefer one doesn't need to
> type `-f ...`.

Yes having to specify the path to the Makefile is an extra hurdle.  As 
the cover letter mentions, the first patch I made was in the top-level 
Makefile[1].

The issue here is that it means that all regular `make` commands
would be going through the `else` branch within a make sub-process
and I wasn't too comfortable with this.  There's every chance that
someone, somewhere is using a particular make version or environment
that will cause subtle differences, or mess things up for someone
keeping an eye on the steps taken by a build.  I couldn't figure out
a way of doing this with zero impact except than having a separate
Makefile and let people decide to opt-in when they want containers.

Then I was only suggesting using a minimalist alias e.g.:

     alias kmake='make -f scripts/Makefile.container'

which would appear to work just like in the original patch.

Defining an alias to run a build inside a container is a lot more
fragile, here it's using standard `make` variables such as $(MAKE),
$(MAKEFLAGS) etc. to keep the commands entirely compatible.

> Put another way, for a user, what is the benefit of having this extra
> way of running in a container? For instance, I could see the benefit
> if different tools had different flags or it was a complicated
> procedure, but I think at least `podman` shares the flags used here.

The advantage here is that it's easier to define your CONTAINER
variable and run make as usual than start an interactive container by
hand with a volume and then run make inside.  It reduces the scope
for differences and makes builds more reproducible, for example you
can just share your command line and others will be using the exact
same toolchain and build environment as you.  This is also to enable
developers to easily reproduce builds from CI bots.  It's been one of
the driving principles behind tuxmake except I'm looking at it from a
neutral point of view here.  In other words, it's a step towards
increasing quality control upstream.

A related topic which was covered at Plumbers is to have first-party
container images, with Containerfiles maintained upstream to
facilitate using the kernel.org toolchains.  It's not a requirement
for this patch but both ideas enhance each other.  An upstream CI bot
using these container images would just have to share i.e. `make
CONTAINER=korg-clang:21`.

> Should this instead be a document inside `Documentation/` somewhere
> that explains how to do this, pitfalls, advanced options, etc. and
> give example command lines for different tools?

Yes, I did think of writing a documentation page alongside this patch
but eventually made the RFC with a cover letter instead to keep it
more as an open discussion.  Any solution to run containerized builds
would need to be documented, even if they're trivial to use.  I think
the Makefile approach is the most elegant one but if others aren't
convinced by it then starting with just some documentation might help
getting to the bottom of this and decide what to do next.

> If we do end up with `CONTAINER=`, then I think it should make it work
> without having to pass `-f ...`, to make it easier. Or, even better,
> like the KUnit script, we could have a script that does the right
> thing and reads a config from the user, so that one can just type
> something like, picking whatever tooling the user configured (e.g.
> Docker vs. Podman, default image, etc.):
> 
>      scripts/container.py defconfig

Right but then I think we would have to deal with the variables
handled by `make` which can be passed either via the environment or
on the command line, so that's similar to the issues with an alias.

We could do something like with Android builds (build/envsetup.sh)
with a file to source:

     . scripts/containerize
     m CONTAINER=korg-clang:21 defconfig

where `m` is just an arbitrary alias name obviously :)

Thank you for your feedback.  I can spend some time investigating
alternative approaches if they seem worthwhile.  I'd be interested to
know what others think of this too.

Thanks,
Guillaume


[1] Initial patch:

diff --git a/Makefile b/Makefile
index c6f549f6a4ae..4cbd8e040db7 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,15 @@
  # SPDX-License-Identifier: GPL-2.0
+ifneq ($(CONTAINER),)
+CONTAINER_COMMAND ?= docker
+PHONY := __all
+__all:
+%:
+       @echo Running in $(CONTAINER_COMMAND) $(CONTAINER)
+       @$(CONTAINER_COMMAND) run -it -v $(PWD):/src -w /src \
+       $(CONTAINER) $(MAKE) \
+       $(subst CONTAINER=$(CONTAINER),,$(MAKEFLAGS)) \
+       $(GNUMAKEFLAGS) $(MAKECMDGOALS)
+else
  VERSION = 6
  PATCHLEVEL = 7
  SUBLEVEL = 0
@@ -2051,3 +2062,4 @@ FORCE:
  # Declare the contents of the PHONY variable as phony.  We keep that
  # information in a variable so we can use it in if_changed and friends.
  .PHONY: $(PHONY)
+endif # CONTAINER

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ