Message-ID: <6d3953f3-14ce-4a58-a018-3636e77dbdf8@arm.com>
Date: Thu, 2 Oct 2025 09:46:29 +0100
From: Suzuki K Poulose <suzuki.poulose@....com>
To: Steven Price <steven.price@....com>, Marc Zyngier <maz@...nel.org>
Cc: kvm@...r.kernel.org, kvmarm@...ts.linux.dev,
 Catalin Marinas <catalin.marinas@....com>, Will Deacon <will@...nel.org>,
 James Morse <james.morse@....com>, Oliver Upton <oliver.upton@...ux.dev>,
 Zenghui Yu <yuzenghui@...wei.com>, linux-arm-kernel@...ts.infradead.org,
 linux-kernel@...r.kernel.org, Joey Gouly <joey.gouly@....com>,
 Alexandru Elisei <alexandru.elisei@....com>,
 Christoffer Dall <christoffer.dall@....com>, Fuad Tabba <tabba@...gle.com>,
 linux-coco@...ts.linux.dev,
 Ganapatrao Kulkarni <gankulkarni@...amperecomputing.com>,
 Gavin Shan <gshan@...hat.com>, Shanker Donthineni <sdonthineni@...dia.com>,
 Alper Gun <alpergun@...gle.com>, "Aneesh Kumar K . V"
 <aneesh.kumar@...nel.org>, Emi Kisanuki <fj0570is@...itsu.com>,
 Vishal Annapurve <vannapurve@...gle.com>
Subject: Re: [PATCH v10 06/43] arm64: RME: Define the user ABI

On 01/10/2025 15:44, Steven Price wrote:
> On 01/10/2025 13:28, Marc Zyngier wrote:
>> On Wed, 20 Aug 2025 15:55:26 +0100,
>> Steven Price <steven.price@....com> wrote:
>>>
>>> There is one (multiplexed) CAP which can be used to create, populate and
>>> then activate the realm.
>>>
>>> Co-developed-by: Suzuki K Poulose <suzuki.poulose@....com>
>>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@....com>
>>> Signed-off-by: Steven Price <steven.price@....com>
>>> Reviewed-by: Gavin Shan <gshan@...hat.com>
>>> ---
>>> Changes since v9:
>>>   * Improvements to documentation.
>>>   * Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
>>> Changes since v8:
>>>   * Minor improvements to documentation following review.
>>>   * Bump the magic numbers to avoid conflicts.
>>> Changes since v7:
>>>   * Add documentation of new ioctls
>>>   * Bump the magic numbers to avoid conflicts
>>> Changes since v6:
>>>   * Rename some of the symbols to make their usage clearer and avoid
>>>     repetition.
>>> Changes from v5:
>>>   * Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
>>>     KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
>>> ---
>>>   Documentation/virt/kvm/api.rst    | 71 +++++++++++++++++++++++++++++++
>>>   arch/arm64/include/uapi/asm/kvm.h | 49 +++++++++++++++++++++
>>>   include/uapi/linux/kvm.h          | 10 +++++
>>>   3 files changed, 130 insertions(+)
>>>
>>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>>> index 6aa40ee05a4a..69c0a9eba6c5 100644
>>> --- a/Documentation/virt/kvm/api.rst
>>> +++ b/Documentation/virt/kvm/api.rst
>>> @@ -3549,6 +3549,11 @@ Possible features:
>>>   	  Depends on KVM_CAP_ARM_EL2_E2H0.
>>>   	  KVM_ARM_VCPU_HAS_EL2 must also be set.
>>>   
>>> +	- KVM_ARM_VCPU_REC: Allocate a REC (Realm Execution Context) for this
>>> +	  VCPU. This must be specified on all VCPUs created in a Realm VM.
>>> +	  Depends on KVM_CAP_ARM_RME.
>>> +	  Requires KVM_ARM_VCPU_FINALIZE(KVM_ARM_VCPU_REC).
>>> +
>>>   4.83 KVM_ARM_PREFERRED_TARGET
>>>   -----------------------------
>>>   
>>> @@ -5122,6 +5127,7 @@ Recognised values for feature:
>>>   
>>>     =====      ===========================================
>>>     arm64      KVM_ARM_VCPU_SVE (requires KVM_CAP_ARM_SVE)
>>> +  arm64      KVM_ARM_VCPU_REC (requires KVM_CAP_ARM_RME)
>>>     =====      ===========================================
>>>   
>>>   Finalizes the configuration of the specified vcpu feature.
>>> @@ -6476,6 +6482,30 @@ the capability to be present.
>>>   
>>>   `flags` must currently be zero.
>>>   
>>> +4.144 KVM_ARM_VCPU_RMM_PSCI_COMPLETE
>>> +------------------------------------
>>> +
>>> +:Capability: KVM_CAP_ARM_RME
>>> +:Architectures: arm64
>>> +:Type: vcpu ioctl
>>> +:Parameters: struct kvm_arm_rmm_psci_complete (in)
>>> +:Returns: 0 if successful, < 0 on error
>>> +
>>> +::
>>> +
>>> +  struct kvm_arm_rmm_psci_complete {
>>> +	__u64 target_mpidr;
>>> +	__u32 psci_status;
>>> +	__u32 padding[3];
>>> +  };
>>> +
>>> +Where PSCI functions are handled by user space, the RMM needs to be informed of
>>> +the target of the operation using `target_mpidr`, along with the status
>>> +(`psci_status`). The RMM v1.0 specification defines two functions that require
>>> +this call: PSCI_CPU_ON and PSCI_AFFINITY_INFO.
>>> +
>>> +If the kernel is handling PSCI then this is done automatically and the VMM
>>> +doesn't need to call this ioctl.
>>
>> Why should userspace involved in this? Why can't this be a
>> notification that the host delivers to the RMM when the vcpu is about
>> to run?
> 
> This is only when PSCI is being handled by user space. If the kernel
> (i.e KVM) is handling PSCI then indeed there's no user space involvement.
> 
> I'm not sure how we could avoid this when PSCI is being implemented in
> user space. Or am I missing something?

I think there is a bit of disconnect here.

The RMM doesn't track the RECs for a given vCPU. So, when it requires
the REC object for a given vCPU, the Host provides this via an 
RMI_PSCI_COMPLETE call. This is used for PSCI_CPU_ON and 
PSCI_AFFINITY_INFO today, where the RMM can do the book keeping
for the REC and emulate the PSCI. Now, the host does have a control
on whether to ACCEPT or REJECT a request (for CPU_ON).
The RMM requires the PSCI_COMPLETE call, before it can return the
PSCI_CPU_ON back to the caller and also before the target vCPU can
run. Thus, this cannot be delayed until the "new VCPU" is run.

Like Steven mentioned, this is only useful in the UABI if the VMM is
handling the PSCI. And this must be issued, before the target vCPU
can be scheduled.

Suzuki

Powered by blists - more mailing lists