linux-kernel - Issues with kexec on arm64

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <CAFpOueSJFyAa2q=0S5Ch6cCy_ORKBV8Y-FUTq6bTky37LJ7M_A@mail.gmail.com>
Date: Tue, 24 Dec 2024 13:36:41 +0200
From: Itai Handler <itai.handler@...il.com>
To: linux-kernel@...r.kernel.org, linux-arm-kernel@...ts.infradead.org
Subject: Issues with kexec on arm64

[Sorry about my previous e-mail on this subject. It got corrupted.
Please ignore it.]

Hello,

I'm encountering kernel panics / system hangs when attempting to
kexec a vmlinux file on arm64 architecture.

It happens both on qemu and on real hardware.

These issues occur on all kernels from v4.19 to the latest mainline.
A sample panic output looks as follows:
  kernel BUG at arch/arm64/mm/mmu.c:217!
  Internal error: Oops - BUG: 00000000f2000800 [#1] SMP
  CPU: 0 PID: 0 Comm: swapper Not tainted 6.6.0 #292
  Hardware name: linux,dummy-virt (DT)
  pstate: 800000c5 (Nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
  pc : __create_pgd_mapping+0xe8/0x3b0
  lr : __create_pgd_mapping+0x44/0x3b0
  sp : fffffe00804d3c20
  x29: fffffe00804d3c20 x28: fffffe0080620000 x27: fffffffefdbc0000
  x26: fffffe0080300000 x25: 0000000040010000 x24: fffffffefdbc8020
  x23: fffffe0080010000 x22: 0000000000000040 x21: fffffe0080010000
  x20: fffffe0080300000 x19: 0040000000000783 x18: 0000000000000000
  x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
  x14: fffffffefdde0000 x13: fffffe00804d3c78 x12: 0000000000001d68
  x11: 0000000000001d64 x10: fffffe00804d3c2c x9 : fffffffefdde0000
  x8 : 0000000040420000 x7 : 0000000000001d68 x6 : 0000000000000000
  x5 : fffffe00a0010000 x4 : 0000000000001004 x3 : fffffe0480010000
  x2 : fffffe00804f7ec0 x1 : 0000000000000000 x0 : 0000000000000000
  Call trace:
   __create_pgd_mapping+0xe8/0x3b0
   map_kernel_segment+0x74/0xb0
   paging_init+0xec/0x4f8
   setup_arch+0x234/0x52c
   start_kernel+0x64/0x500
   __primary_switched+0xb4/0xbc
  Code: f9400300 92400400 f1000c1f 54000060 (d4210000)
  ---[ end trace 0000000000000000 ]---
  Kernel panic - not syncing: Oops - BUG: Fatal exception

I bisected those panics to 8eb7e28d4c642c310f25c18f80a44dd4b01c694e
("arm64/mm: move runtime pgds to rodata"), which was added on v4.19.

I also reconstructed the full call trace (by adding "noinline" to the
relevant functions):
  alloc_init_cont_pte+0x6c/0x1e0
  init_pmd+0x154/0x1c8
  alloc_init_cont_pmd+0x11c/0x174
  alloc_init_pud+0xc4/0x148
  __create_pgd_mapping+0xa8/0x130
  map_kernel_segment+0xc8/0x168
  map_kernel+0x98/0x1a8
  paging_init+0x7c/0x418
  setup_arch+0x224/0x570
  start_kernel+0x5c/0x4f0

My understanding is that the panic occurs inside alloc_init_cont_pte,
at the BUG_ON(pmd_bad(..)) line.

kexec-tools version: 2.0.29
qemu version: 8.1.94 (v8.2.0-rc4)

The .config files are created using the following script:
  make tinyconfig
  # --- Options to enable ---
  scripts/config --enable CONFIG_EXPERT
  scripts/config --enable CONFIG_TTY
  scripts/config --enable CONFIG_PRINTK
  scripts/config --enable CONFIG_BUG
  scripts/config --enable CONFIG_STACKTRACE
  scripts/config --enable CONFIG_BINFMT_ELF
  scripts/config --enable CONFIG_BINFMT_SCRIPT
  scripts/config --enable CONFIG_PROC_FS
  scripts/config --enable CONFIG_BLOCK
  scripts/config --enable CONFIG_BLK_DEV
  scripts/config --enable CONFIG_BLK_DEV_NULL_BLK
  scripts/config --enable CONFIG_BLK_DEV_INITRD
  scripts/config --enable CONFIG_PANIC_ON_OOPS
  scripts/config --enable CONFIG_DEVTMPFS
  scripts/config --enable CONFIG_DEVTMPFS_MOUNT
  scripts/config --enable CONFIG_NET
  scripts/config --enable CONFIG_PCI
  scripts/config --enable CONFIG_PCI_HOST_GENERIC
  scripts/config --enable CONFIG_VIRTIO_MENU
  scripts/config --enable CONFIG_VIRTIO_BLK
  scripts/config --enable CONFIG_VIRTIO_PCI
  scripts/config --enable CONFIG_NET_9P
  scripts/config --enable CONFIG_NET_9P_VIRTIO
  scripts/config --enable CONFIG_9P_FS
  scripts/config --enable CONFIG_CONFIGFS_FS
  scripts/config --enable CONFIG_SUSPEND
  scripts/config --enable CONFIG_PROC_KCORE
  scripts/config --enable CONFIG_KEXEC
  scripts/config --enable CONFIG_SERIAL_AMBA_PL011
  scripts/config --enable CONFIG_SERIAL_AMBA_PL011_CONSOLE
  scripts/config --enable CONFIG_POSIX_TIMERS
  scripts/config --enable CONFIG_KALLSYMS
  scripts/config --enable CONFIG_ARM64_64K_PAGES
  # --- Options to disable ---
  scripts/config --disable CONFIG_IPV6
  scripts/config --disable CONFIG_WIRELESS
  scripts/config --disable CONFIG_SWAP
  make olddefconfig

To run the qemu VM I use the following script:
  APPEND="earlycon console=ttyAMA0 loglevel=8"
  qemu-system-aarch64 \
    -M virt \
    -cpu cortex-a53 \
    -smp 4 \
    -m 4096 \
    -kernel ~/vmshare/Image \
    -initrd ~/vmshare/rootfs.cpio \
    -nographic \
    -append "${APPEND}" \
    -fsdev local,id=vmshare,path=$HOME/vmshare,security_model=mapped,multidevs=remap
\
    -device virtio-9p-pci,fsdev=vmshare,mount_tag=vmshare \

I built the root filesystem using buildroot 2024.08.2, using the following
defconfig:
  BR2_aarch64=y
  BR2_ARM64_PAGE_SIZE_64K=y
  BR2_KERNEL_HEADERS_4_19=y
  BR2_PACKAGE_HOST_GDB=y
  BR2_GDB_VERSION_15=y
  BR2_PACKAGE_KEXEC=y
  BR2_PACKAGE_KEXEC_ZLIB=y
  BR2_TARGET_ROOTFS_CPIO=y
  BR2_PACKAGE_HOST_KMOD=y

To kexec the file I use the following command:
  kexec -d -c -l /media/vmshare/vmlinux \
    --initrd=/media/vmshare/rootfs.cpio \
    --reuse-cmdline \
    && kexec -d -e

Thanks,
Itai Handler