linux-kernel - Re: v6.14-12245-g91e5bfe317d8: Boot regression: rk3399-rock-pi-4b dragonboard-410c dragonboard-845c no console output

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250403011849.GA3138383@ax162>
Date: Wed, 2 Apr 2025 18:18:49 -0700
From: Nathan Chancellor <nathan@...nel.org>
To: Naresh Kamboju <naresh.kamboju@...aro.org>
Cc: open list <linux-kernel@...r.kernel.org>,
	Linux ARM <linux-arm-kernel@...ts.infradead.org>,
	lkft-triage@...ts.linaro.org,
	Linux Regressions <regressions@...ts.linux.dev>,
	clang-built-linux <llvm@...ts.linux.dev>,
	Arnd Bergmann <arnd@...db.de>, Ard Biesheuvel <ardb@...nel.org>,
	Dan Carpenter <dan.carpenter@...aro.org>,
	Anders Roxell <anders.roxell@...aro.org>
Subject: Re: v6.14-12245-g91e5bfe317d8: Boot regression: rk3399-rock-pi-4b
 dragonboard-410c dragonboard-845c no console output

Hi Naresh,

On Wed, Apr 02, 2025 at 06:19:19PM +0530, Naresh Kamboju wrote:
> Regressions on rk3399-rock-pi-4b, dragonboard-410c and dragonboard-845c
> the lto-thing, hardening and lto-full config boot failed with toolchain
> clang-nightly on the mainline master branch with no console output.
> 
> First seen on the v6.14-12245-g91e5bfe317d8
>  Good: v6.14-11270-g08733088b566
>  Bad: v6.14-12245-g91e5bfe317d8
> 
> Regressions found on rk3399-rock-pi-4b:
>   - boot/clang-nightly-lkftconfig-kselftest
>   - boot/clang-nightly-lkftconfig-lto-thing
>   - boot/clang-nightly-lkftconfig-hardening
>   - boot/clang-nightly-lkftconfig-lto-full
> 
> Regressions found on dragonboard-410c:
>   - boot/clang-nightly-lkftconfig-lto-thing
>   - boot/clang-nightly-lkftconfig-lto-full
>   - boot/clang-nightly-lkftconfig-hardening
> 
> Regressions found on dragonboard-845c:
>   - boot/clang-nightly-lkftconfig-hardening
>   - boot/clang-nightly-lkftconfig-lto-thing
> 
> Regression Analysis:
>  - New regression? Yes
>  - Reproducibility? Yes
> 
> Boot regression: rk3399-rock-pi-4b dragonboard-410c dragonboard-845c
> no console output
> 
> Reported-by: Linux Kernel Functional Testing <lkft@...aro.org>
> 
> ## Boot log
> Starting kernel
> ...
> <No console output>
> 
> 
> ## Source
> * Kernel version: 6.14.0
> * Git tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> * Git sha: 91e5bfe317d8f8471fbaa3e70cf66cae1314a516
> * Git describe: v6.14-12245-g91e5bfe317d8
> * Project details:
> https://qa-reports.linaro.org/lkft/linux-mainline-master/build/v6.14-12245-g91e5bfe317d8/
> * Architectures: arm64 (rk3399-rock-pi-4b, dragonboard-410c, dragonboard-845c)
> * Toolchains: clang-nightly (Debian clang version 21.0.0 )

The version string for the toolchain seems to be slightly truncated but
from the configuration it is:

  Debian clang version 21.0.0 (++20250330105456+3b3d1a5c2614-1~exp1~20250330225508.1357)

which can be parsed with our parse-debian-clang.py [1] to get more
information:

  $ scripts/parse-debian-clang.py -p -v 'Debian clang version 21.0.0 (++20250330105456+3b3d1a5c2614-1~exp1~20250330225508.1357)'
  clang checkout date: 2025-03-30 10:54 UTC (3 days, 14:17:24.134272 ago)
  clang revision: 3b3d1a5c2614
  clang revision link: https://github.com/llvm/llvm-project/commit/3b3d1a5c2614

Our CI is using a slightly newer version:

  $ scripts/parse-debian-clang.py -p -v 'Debian clang version 21.0.0 (++20250401112529+290d7b82cb5d-1~exp1~20250401112547.1360)'
  clang checkout date: 2025-04-01 11:25 UTC (1 day, 13:49:39.394836 ago)
  clang revision: 290d7b82cb5d
  clang revision link: https://github.com/llvm/llvm-project/commit/290d7b82cb5d

Can you see if it is reproducible with that revision? For what it's
worth, both of the arm64 boxes I have can boot a ThinLTO kernel compiled
with a version of LLVM @ 749535ba2808e133682074f712ac6829335f8875, so it
could be something that was broken for a little bit but Debian happened
to sync before the fix was committed.

[1]: https://github.com/ClangBuiltLinux/continuous-integration2/blob/fe4844afc1be91d469fc162c8a179f23fafb9384/scripts/parse-debian-clang.py

Cheers,
Nathan