Message-Id: <20251117-rneri-wakeup-mailbox-v7-0-4a8b82ab7c2c@linux.intel.com>
Date: Mon, 17 Nov 2025 09:02:46 -0800
From: Ricardo Neri <ricardo.neri-calderon@...ux.intel.com>
To: x86@...nel.org, Krzysztof Kozlowski <krzk+dt@...nel.org>, 
 Conor Dooley <conor+dt@...nel.org>, Rob Herring <robh@...nel.org>, 
 "K. Y. Srinivasan" <kys@...rosoft.com>, 
 Haiyang Zhang <haiyangz@...rosoft.com>, Wei Liu <wei.liu@...nel.org>, 
 Dexuan Cui <decui@...rosoft.com>, Michael Kelley <mhklinux@...look.com>, 
 "Rafael J. Wysocki" <rafael@...nel.org>
Cc: Saurabh Sengar <ssengar@...ux.microsoft.com>, 
 Chris Oo <cho@...rosoft.com>, "Kirill A. Shutemov" <kas@...nel.org>, 
 linux-hyperv@...r.kernel.org, devicetree@...r.kernel.org, 
 linux-acpi@...r.kernel.org, linux-kernel@...r.kernel.org, 
 Ricardo Neri <ricardo.neri@...el.com>, 
 Ricardo Neri <ricardo.neri-calderon@...ux.intel.com>, 
 "Rafael J. Wysocki (Intel)" <rafael.j.wysocki@...el.com>, 
 Yunhong Jiang <yunhong.jiang@...ux.intel.com>, 
 Thomas Gleixner <tglx@...utronix.de>
Subject: [PATCH v7 0/9] x86/hyperv/hv_vtl: Use a wakeup mailbox to boot
 secondary CPUs

Hi,

Many thanks to Boris, Rafael, Rob, and Dexuan for their valuable feedback!
The main change in this version is the removal of the patch that moved the
ACPI mailbox code from the x86 ACPI subsystem to a generic location. Users
with DeviceTree-based firmware who wish use the ACPI wakeup mailbox need to
select CONFIG_ACPI=y.

I dropped the Reviewed-by and Acked-by tags from patches 1, 7, and 8 as I
made non-trivial changes to them. Once reviewed, maybe the patchset is
ready to be merged?

I made minor changes to the cover letter to reflect the contents of this
updated version.

Thanks in advance for your feedback!

...

This patchset adds functionality to use the ACPI wakeup mailbox to boot
secondary CPUs in Hyper-V VTL level 2 TDX guests with DeviceTree-based
virtual firmware. Although this is the target use case, the use of the
mailbox depends solely on it being enumerated in the DeviceTree graph.

On x86 platforms, secondary CPUs are typically booted using INIT assert,
de-assert followed by Start-Up IPI messages. Virtual machines can also use
hypercalls to bring up secondary CPUs to a desired execution state. These
two mechanisms require support from the hypervisor. Confidential computing
VMs in a TDX environment cannot use this mechanism because the hypervisor
is considered an untrusted entity.

Linux already supports the ACPI Multiprocessor Wakeup Structure in which
the guest platform firmware boots the secondary CPUs and transfers control
to the kernel using a mailbox. This mechanism does not need involvement
of the VMM. It can be used in a Hyper-V VTL level 2 TDX guest.

Currently, this mechanism can only be used on x86 platforms with firmware
that supports ACPI. There are platforms that use DeviceTree (e.g., OpenHCL
[2]) instead of ACPI to describe the hardware.

Provided that the wakeup mailbox enumerated in a DeviceTree-based platform
firmware is implemented as described in the ACPI specification, the kernel
can use the existing ACPI code for both DeviceTree and ACPI systems. The
DeviceTree firmware does not need to use any ACPI table to enumerate the
mailbox.

This patchset is structured as follows:

   * Expose functions to reuse the code handling the ACPI Multiprocessor
     Wakeup Structure outside of ACPI code. (patch 1)
   * Define DeviceTree bindings to enumerate a mailbox as described in
     the ACPI specification. (patch 2)
   * Find and set up the wakeup mailbox if enumerated in the DeviceTree
     graph. (patch 3)
   * Prepare Hyper-V VTL2 TDX guests to use the Wakeup Mailbox to boot
     secondary CPUs when available. (patches 4-9)

I have tested this patchset on a Hyper-V host with VTL2 OpenHCL, QEMU, and
physical hardware.

Changes in v7:
- Dropped the patch that relocated the ACPI wakeup mailbox to an generic
  location. (Boris)
- Instead, added function declarations to use the wakeup mailbox from
  outside ACPI code. Also added stubs for !CONFIG_ACPI.
- Link to v6: https://lore.kernel.org/r/20251016-rneri-wakeup-mailbox-v6-0-40435fb9305e@linux.intel.com

Changes in v6:
- Fixed a build error with !CONFIG_X86_MAILBOX_WAKEUP and
  CONFIG_HYPER_VTL_MODE.
- Added Acked-by tags from Rafael. Thanks!
- Added Reviewed-by tags from Dexuan and Rob. Thanks!
- Corrected typos and function names in the changelog.
- Link to v5: https://lore.kernel.org/r/20250627-rneri-wakeup-mailbox-v5-0-df547b1d196e@linux.intel.com

Changes in v5:
- Referred in the DeviceTree binding documentation the section and
  section of the ACPI specification that defines the wakeup mailbox.
- Moved the dependency on CONFIG_OF to patch 4, where the flattened
  DeviceTree is parsed for the mailbox.
- Fixed a warning from yamllint regarding line lengths.
- Link to v4: https://lore.kernel.org/r/20250603-rneri-wakeup-mailbox-v4-0-d533272b7232@linux.intel.com

Changes in v4:
- Added Reviewed-by: tags from Michael Kelley. Thanks!
- Relocated the common wakeup code from acpi/madt_wakeup.c to a new
  smpwakeup.c to be used in DeviceTree- and ACPI-based systems.
- Dropped the x86 CPU bindings as they are not a good fit to document
  firmware features.
- Dropped the code that parsed and validated of the `enable-method`
  property for cpu@N nodes in x86. Instead, unconditionally parse and use
  the wakeup mailbox when found.
- Updated the wakeup mailbox schema to avoid redefing the structure and
  operation of the mailbox. Instead, refer to the ACPI specification.
  Also clarified that the enumeration of the mailbox is done separately.
- Prefixed helper functions of wakeup code with acpi_.
- Link to v3: https://lore.kernel.org/r/20250503191515.24041-1-ricardo.neri-calderon@linux.intel.com

Changes in v3:
- Only move out of the acpi directory acpi_wakeup_cpu() and its
  accessory variables. Use helper functions to access the mailbox as
  needed. This also fixed the warnings about unused code with CONFIG_
  ACPI=n that Michael reported.
- Major rework of the DeviceTree bindings and schema. Now there is a
  reserved-memory binding for the mailbox as well as a new x86 CPU
  bindings. Both have `compatible` properties.
- Rework of the code parsing the DeviceTree bindings for the mailbox.
  Now configuring the mailbox depends solely on its enumeration in the
  DeviceTree and not on Hyper-V VTL2 TDX guest.
- Do not make reserving the first 1MB of memory optional. It is not
  needed and may introduce bugs.
- Prepare Hyper-V VTL2 guests to unconditionally use the mailbox in TDX
  environments. If the mailbox is not available, booting secondary CPUs
  will fail gracefully.
- Link to v2: https://lore.kernel.org/r/20240823232327.2408869-1-yunhong.jiang@linux.intel.com

Changes in v2:
- Fix the cover letter's summary phrase.
- Fix the DT binding document to pass validation.
- Change the DT binding document to be ACPI independent.
- Move ACPI-only functions into the #ifdef CONFIG_ACPI.
- Change dtb_parse_mp_wake() to return mailbox physical address.
- Rework the hv_is_private_mmio_tdx().
- Remove unrelated real mode change from the patch that marks mailbox
  page private.
- Check hv_isolation_type_tdx() instead of wakeup_mailbox_addr in
  hv_vtl_init_platform() because wakeup_mailbox_addr is not parsed yet.
- Add memory range support to reserve_real_mode.
- Remove realmode_reserve callback and use the memory range.
- Move setting the real_mode_header to hv_vtl_init_platform.
- Update comments and commit messages.
- Minor style changes.
- Link to v1: https://lore.kernel.org/r/20240806221237.1634126-1-yunhong.jiang@linux.intel.com

[1]. https://lore.kernel.org/all/aNxGzWMoM_oQ6n1N@liuwe-devbox-ubuntu-v2.lamzopl0uupeniq2etz1fddiyg.xx.internal.cloudapp.net/
[2]. https://openvmm.dev/guide/user_guide/openhcl.html
--
2.43.0

---
Ricardo Neri (5):
      x86/acpi: Add functions to setup and access the wakeup mailbox
      dt-bindings: reserved-memory: Wakeup Mailbox for Intel processors
      x86/dt: Parse the Wakeup Mailbox for Intel processors
      x86/acpi: Add a helper get the address of the wakeup mailbox
      x86/hyperv/vtl: Use the wakeup mailbox to boot secondary CPUs

Yunhong Jiang (4):
      x86/hyperv/vtl: Set real_mode_header in hv_vtl_init_platform()
      x86/realmode: Make the location of the trampoline configurable
      x86/hyperv/vtl: Setup the 64-bit trampoline for TDX guests
      x86/hyperv/vtl: Mark the wakeup mailbox page as private

 .../reserved-memory/intel,wakeup-mailbox.yaml      | 50 ++++++++++++++++++++++
 arch/x86/hyperv/hv_vtl.c                           | 36 ++++++++++++++--
 arch/x86/include/asm/acpi.h                        | 16 +++++++
 arch/x86/include/asm/x86_init.h                    |  3 ++
 arch/x86/kernel/acpi/madt_wakeup.c                 | 16 +++++++
 arch/x86/kernel/devicetree.c                       | 47 ++++++++++++++++++++
 arch/x86/kernel/x86_init.c                         |  3 ++
 arch/x86/realmode/init.c                           |  7 ++-
 8 files changed, 170 insertions(+), 8 deletions(-)
---
base-commit: 6f85aad74a70d17919a64ecd93037aa51c08698d
change-id: 20250602-rneri-wakeup-mailbox-328efe72803f

Best regards,
-- 
Ricardo Neri <ricardo.neri-calderon@...ux.intel.com>

Powered by blists - more mailing lists