Message-ID: <aQSKpZDrLzf/bcx7@e129823.arm.com>
Date: Fri, 31 Oct 2025 10:08:37 +0000
From: Yeoreum Yun <yeoreum.yun@....com>
To: Sebastian Ene <sebastianene@...gle.com>
Cc: maz@...nel.org, oliver.upton@...ux.dev, joey.gouly@....com,
	suzuki.poulose@....com, yuzenghui@...wei.com,
	catalin.marinas@....com, will@...nel.org, perlarsen@...gle.com,
	ayrton@...gle.com, linux-arm-kernel@...ts.infradead.org,
	kvmarm@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 1/2] KVM: arm64: fix FF-A call failure when ff-a driver
 is built-in

Hi Sebastian,

> > Until has_version_negotiated is set to true,
> > all FF-A function calls fail except FFA_VERSION.
> > The has_version_negotiated flag is set to true when
> > the first FFA_VERSION call is made after init_hyp_mode().
> >
> > This works fine when the FF-A driver is built as a module,
> > since ffa_init() is invoked after kvm_arm_init(), allowing do_ffa_version()
> > to set has_version_negotiated to true.
> >
> > However, when the FF-A driver is built-in (CONFIG_ARM_FFA_TRANSPORT=y),
> > all FF-A calls fail. This happens because ffa_init() runs before
> > kvm_arm_init() — the init level of ffa_init() is rootfs_initcall.
> > As a result, the hypervisor cannot set has_version_negotiated,
> > since the FFA_VERSION call made in ffa_init() does not trap to the hypervisor
> > (HCR_EL2.TSC is cleared before kvm_arm_init()).
> >
>
> I understand the reason behind the patch but this is problematic to have
> the builtin driver load before pKVM because the hypervisor would be
> un-aware of the host mapped buffers. (eg. the call from ffa_rxtx_map is
> not trapped because it is too early). Essentially, you will end up
> bypassing the hyp FF-A proxy which I think you will want to avoid.

Ah. I've overlooed the ffa_rxtx_map proxy.
But unfortunately, some of depndency with the driver using arm_ffa
driver, ffa_init() should be called first then other drivers' initcall
(usually, these kind of driver defines its one initcall with
device_initcall()) (i.e) https://lore.kernel.org/all/20250618102302.2379029-1-yeoreum.yun@arm.com/.

Though I arm_ffa driver provide an API getting mapped rx/tx buffer,
But this seems to reverse dependency -- kvm depends on arm_ffa driver.

I’ve been thinking about some possible solutions,
but in my narrow idea, valid solution is kvm_arm_init() as
subsys_initcall_sync() and call kvm_init() in module_init() like
attached modification.

Do you have any idea?

Thanks.

------<&--------

diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 870953b4a8a7..44711f3c7e04 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -65,7 +65,7 @@ DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);

 DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt);

-static bool vgic_present, kvm_arm_initialised;
+static bool vgic_present, in_hyp_mode, okvm_arm_early_initialised, kvm_arm_initialised;

 static DEFINE_PER_CPU(unsigned char, kvm_hyp_initialized);

@@ -2827,11 +2827,42 @@ void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
        kvm_arm_resume_guest(irqfd->kvm);
 }

+static __init int kvm_arm_post_init(void)
+{
+       int err;
+
+       if (!kvm_arm_early_initialised)
+               return -ENODEV;
+
+       /*
+        * FIXME: Do something reasonable if kvm_init() fails after pKVM
+        * hypervisor protection is finalized.
+        */
+       err = kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+       if (err) {
+               teardown_subsystems();
+               if (!in_hyp_mode)
+                       teardown_hyp_mode();
+               kvm_arm_vmid_alloc_free();
+               return err;
+       }
+
+       /*
+        * This should be called after initialization is done and failure isn't
+        * possible anymore.
+        */
+       if (!in_hyp_mode)
+               finalize_init_hyp_mode();
+
+       kvm_arm_initialised = true;
+       return 0;
+}
+
+
 /* Initialize Hyp-mode and memory mappings on all CPUs */
 static __init int kvm_arm_init(void)
 {
        int err;
-       bool in_hyp_mode;

        if (!is_hyp_mode_available()) {
                kvm_info("HYP mode not available\n");
@@ -2893,30 +2924,18 @@ static __init int kvm_arm_init(void)
                                     "h" : "n"),
                 cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) ? "+NV2": "");

-       /*
-        * FIXME: Do something reasonable if kvm_init() fails after pKVM
-        * hypervisor protection is finalized.
-        */
-       err = kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE);
-       if (err)
-               goto out_subs;
-
-       /*
-        * This should be called after initialization is done and failure isn't
-        * possible anymore.
-        */
-       if (!in_hyp_mode)
-               finalize_init_hyp_mode();
-
-       kvm_arm_initialised = true;
+       kvm_arm_early_initialised = true;

-       return 0;
+#ifdef MODULE
+       err = kvm_arm_post_init();
+#endif
+       return err;

-out_subs:
-       teardown_subsystems();
 out_hyp:
-       if (!in_hyp_mode)
+       if (!in_hyp_mode) {
                teardown_hyp_mode();
+               in_hyp_mode = false;
+       }
 out_err:
        kvm_arm_vmid_alloc_free();
        return err;
@@ -2995,4 +3014,7 @@ enum kvm_mode kvm_get_mode(void)
        return kvm_mode;
 }

-module_init(kvm_arm_init);
+subsys_initcall_sync(kvm_arm_init);
+#ifndef MODULE
+module_init(kvm_arm_post_init);
+#endif

[...]

--
Sincerely,
Yeoreum Yun

Powered by blists - more mailing lists