linux-kernel - Re: [PATCH v4 0/4] Don't create Python bytecode when building the kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20250426205708.4f90a83d@sal.lan>
Date: Sat, 26 Apr 2025 20:57:08 +0800
From: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To: Akira Yokosawa <akiyks@...il.com>
Cc: Andy Shevchenko <andriy.shevchenko@...el.com>, airlied@...il.com,
 corbet@....net, dri-devel@...ts.freedesktop.org,
 intel-gfx@...ts.freedesktop.org, jani.nikula@...ux.intel.com,
 joonas.lahtinen@...ux.intel.com, linux-doc@...r.kernel.org,
 linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
 maarten.lankhorst@...ux.intel.com, masahiroy@...nel.org,
 mripard@...nel.org, nathan@...nel.org, nicolas.schier@...ux.dev,
 rodrigo.vivi@...el.com, simona@...ll.ch, tursulin@...ulin.net,
 tzimmermann@...e.de
Subject: Re: [PATCH v4 0/4] Don't create Python bytecode when building the
 kernel

Hi Akira,

Em Sat, 26 Apr 2025 11:39:05 +0900
Akira Yokosawa <akiyks@...il.com> escreveu:

> Bothering with might-become-incompatilbe-in-the-future python environment
> variables in kernel Makefiles looks over-engineering to me.
> Also, as Mauro says in 3/4, it is incomplete in that it does not cover
> the cases where those scripts are invoked outside of kernel build.
> And it will interfere with existing developers who want the benefit of
> bytecode caching.
> 
> I'm not precluding the possibility of incoherent bytecode cache; for example
> by using a shared kernel source tree among several developers, and only
> one of them (owner) has a write permission of it.  In that case, said
> owner might update the tree without running relevant python scripts.
> 
> I don't know if python can notice outdated cache and disregard it.
> 
> In such a situation, setting PYTHONPYCACHEPREFIX as an environment
> variable should help, for sure, but only in such special cases.
> 
> Andy, what do you say if I ask reverts of 1/4, 2/4/, and 3/4?

Patches 1 and 2 are, IMO, needed anyway, as they fix a problem:
KERNELDOC environment is not used consistently.

Now, patch 3 is the one that may require more thinking.

I agree with Andy that, when O=<dir> is used, nothing shall be
written to source dir.

There are a couple of reasons for that:

1. source dir may be read only;
2. one may want to do cross compilation and use multiple output
   directories, one for each version;
3. the source dir could be mapped via NFS to multiple machines
   with different architectures.

For (3), it could mean that multiple machines may have different
Python versions, so, sharing the Python bytecode from source dir doesn't
sound a good idea. Also, I'm not sure if the pyc from different archs
would be identical.

With that, there are two options:

a. disable cache;
b. set PYTHONCACHEPREFIX.

We're currently doing (a). I guess everybody agrees that this is
is not ideal.

So, ideally, we should move to (b). For Spinx, the easiest solution
is just to place it under Documentation/output, but this is not
generic enough: ideally, we should revert patch 3 and set
PYTHONCACHEPREFIX when O is used. Eventually, we can apply my
patch for Documentation/output, while we craft such logic.

Regards,
Mauro