Message-ID: <CAAhSdy0MBcD29fNSRgfckXuNmptW_borO7gtVqWvXRSnPBpmJg@mail.gmail.com>
Date: Fri, 30 May 2025 17:15:40 +0530
From: Anup Patel <anup@...infault.org>
To: Andrew Jones <ajones@...tanamicro.com>
Cc: Atish Patra <atish.patra@...ux.dev>, Radim Krčmář <rkrcmar@...tanamicro.com>, 
	Will Deacon <will@...nel.org>, Mark Rutland <mark.rutland@....com>, 
	Paul Walmsley <paul.walmsley@...ive.com>, Palmer Dabbelt <palmer@...belt.com>, 
	Mayuresh Chitale <mchitale@...tanamicro.com>, linux-riscv@...ts.infradead.org, 
	linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org, 
	kvm@...r.kernel.org, kvm-riscv@...ts.infradead.org, 
	linux-riscv <linux-riscv-bounces@...ts.infradead.org>
Subject: Re: [PATCH v3 9/9] RISC-V: KVM: Upgrade the supported SBI version to 3.0

On Fri, May 30, 2025 at 12:44 AM Andrew Jones <ajones@...tanamicro.com> wrote:
>
> On Thu, May 29, 2025 at 11:44:38AM -0700, Atish Patra wrote:
> >
> > On 5/29/25 3:24 AM, Radim Krčmář wrote:
> > > I originally gave up on the idea, but I feel kinda bad for Drew now, so
> > > trying again:
> >
> > I am sorry if some of my replies came across in the wrong way. That was
> > never
> > the intention.
>
> Not at all. Radim only meant that I was defending his patches, even though
> he wasn't :-)
>
> >
> >
> > > 2025-05-28T12:21:59-07:00, Atish Patra <atish.patra@...ux.dev>:
> > > > On 5/28/25 8:09 AM, Andrew Jones wrote:
> > > > > On Wed, May 28, 2025 at 07:16:11AM -0700, Atish Patra wrote:
> > > > > > On 5/26/25 4:13 AM, Andrew Jones wrote:
> > > > > > > On Mon, May 26, 2025 at 11:00:30AM +0200, Radim Krčmář wrote:
> > > > > > > > 2025-05-23T10:16:11-07:00, Atish Patra <atish.patra@...ux.dev>:
> > > > > > > > > On 5/23/25 6:31 AM, Radim Krčmář wrote:
> > > > > > > > > > 2025-05-22T12:03:43-07:00, Atish Patra <atishp@...osinc.com>:
> > > > > > > > > > > Upgrade the SBI version to v3.0 so that corresponding features
> > > > > > > > > > > can be enabled in the guest.
> > > > > > > > > > >
> > > > > > > > > > > Signed-off-by: Atish Patra <atishp@...osinc.com>
> > > > > > > > > > > ---
> > > > > > > > > > > diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm/kvm_vcpu_sbi.h
> > > > > > > > > > > -#define KVM_SBI_VERSION_MAJOR 2
> > > > > > > > > > > +#define KVM_SBI_VERSION_MAJOR 3
> > > > > > > > > > I think it's time to add versioning to KVM SBI implementation.
> > > > > > > > > > Userspace should be able to select the desired SBI version and KVM would
> > > > > > > > > > tell the guest that newer features are not supported.
> > > > > > > We need new code for this, but it's a good idea.
> > > > > > >
> > > > > > > > > We can achieve that through onereg interface by disabling individual SBI
> > > > > > > > > extensions.
> > > > > > > > > We can extend the existing onereg interface to disable a specific SBI
> > > > > > > > > version directly
> > > > > > > > > instead of individual ones to save those IOCTL as well.
> > > > > > > > Yes, I am all in favor of letting userspace provide all values in the
> > > > > > > > BASE extension.
> > > > > > We already support vendorid/archid/impid through one reg. I think we just
> > > > > > need to add the SBI version support to that so that user space can set it.
> > > > > >
> > > > > > > This is covered by your recent patch that provides userspace_sbi.
> > > > > > Why do we need to invent new IOCTL for this ? Once the user space sets the
> > > > > > SBI version, KVM can enforce it.
> > > > > If an SBI spec version provides an extension that can be emulated by
> > > > > userspace, then userspace could choose to advertise that spec version,
> > > > > implement a BASE probe function that advertises the extension, and
> > > > > implement the extension, even if the KVM version running is older
> > > > > and unaware of it. But, in order to do that, we need KVM to exit to
> > > > > userspace for all unknown SBI calls and to allow BASE to be overridden
> > > > You mean only the version field in BASE - Correct ?
> > > No, "BASE probe function" is the sbi_probe_extension() ecall.
> > >
> > > > > by userspace. The new KVM CAP ioctl allows opting into that new behavior.
> > > > But why we need a new IOCTL for that ? We can achieve that with existing
> > > > one reg interface with improvements.
> > > It's an existing IOCTL with a new data payload, but I can easily use
> > > ONE_REG if you want to do everything through that.
> > >
> > > KVM doesn't really need any other IOCTL than ONE_REGs, it's just
> > > sometimes more reasonable to use a different IOCTL, like ENABLE_CAP.
> > >
> > > > > The old KVM with new VMM configuration isn't totally far-fetched. While
> > > > > host kernels tend to get updated regularly to include security fixes,
> > > > > enterprise kernels tend to stop adding features at some point in order
> > > > > to maximize stability. While enterprise VMMs would also eventually stop
> > > > > adding features, enterprise consumers are always free to use their own
> > > > > VMMs (at their own risk). So, there's a real chance we could have
> > > > I think we are years away from that happening (if it happens). My
> > > > suggestion was not to
> > > > try to build a world where no body lives ;). When we get to that
> > > > scenario, the default KVM
> > > > shipped will have many extension implemented. So there won't be much
> > > > advantage to
> > > > reimplement them in the user space. We can also take an informed
> > > > decision at that time
> > > > if the current selective forwarding approach is better
> > > Please don't repeat the design of SUSP/SRST/DBCN.
> > > Seeing them is one of the reasons why I proposed the new interface.
> > >
> > > "Blindly" forwarding DBCN to userspace is even a minor optimization. :)
> > >
> > > >                                                         or we need to
> > > > blindly forward any
> > > > unknown SBI calls to the user space.
> > > Yes, KVM has to do what userpace configures it to do.
> > >
> > > I don't think that implementing unsupported SBI extensions in KVM is
> > > important -- they should not be a hot path.
> > >
> > > > > deployments with older, stable KVM where users want to enable later SBI
> > > > > extensions, and, in some cases, that should be possible by just updating
> > > > > the VMM -- but only if KVM is only acting as an SBI implementation
> > > > > accelerator and not as a userspace SBI implementation gatekeeper.
> > > > But some of the SBI extensions are so fundamental that it must be
> > > > implemented in KVM
> > > > for various reasons pointed by Anup on other thread.
> > > No, SBI does not have to be implemented in KVM at all.
> > >
> > > We do have a deep disagreement on what is virtualization and the role of
> > > KVM in it.  I think that userspace wants a generic ISA accelerator.
> >
> > I think the disagreement is the role of SBI in KVM virtualization rather
> > than
> > a generic virtualization and the role of KVM in it. I completely agree that
> > KVM should act as an accelerator and defer the control to the user space in
> > most of the cases
> > such e.g I/O operations or system related functionalities. However, SBI
> > specification solves
> > much wider problems than those. Broadly we can categorize SBI
> > functionalities into the following
> > areas
> >
> > 1. Bridging ISA GAP
> > 2. Higher Privilege Assistance
> > 3. Virtualization
> > 4. Platform abstraction
> > 5. Confidential computing
> >
> > For #1, #3 and #5, I believe user space shouldn't be involved in
> > implementation
> > some of them are in hot path as well.
>
> IMO, userspace should still be in control of whether or not it's involved
> in #1, #3, and #5. It may make little sense for it to be involved, but the
> choice should still be its.
>
> > For #4 and #2, there are some
> > opportunities which
> > can be implemented in user space depending on the exact need. I am still not
> > clear what is the exact
> > motivation /right now/ to pursue such a path. May be I missed something.
> > As per my understanding from our discussion threads, there are two use cases
> > possible
> >
> > 1. userspace wants to update more states in HSM. What are the states user
> > space should care about scounteren (fixed already in usptream) ?
> > 2. VMM vs KVM version difference - this may be true in the future depending
> > on the speed of RISC-V virtualization adoption in the industry.
> > But we are definitely not there yet. Please let me know if I misunderstood
> > any use cases.
>
> That's what I'm aware of as well, but I see giving userspace back full
> control of what gets accelerated by KVM, and what doesn't, as a fix, which
> is why I wouldn't want to delay it any longer.
>
> >
> > > Even if userspace wants SBI for the M-mode interface, security minded
> > This is probably a 3rd one ? Why we want M-mode interface in the user space
> > ?
> > > userspace aims for as little kernel code as possible.
> >
> > We trust VMM code more than KVM code ?
>
> We should be skeptical of both, which is why we'd rather put as much code
> in userspace as possible. Insecure/faulty userspace will hopefully have
> exploits/bugs contained to the single process. An insecure/faulty KVM
> means the host is compromised/crashed. On x86, Google put a lot of effort
> into moving instruction emulation out of KVM for security concerns[1]. In
> general, if it's not a hot path and there's a way to do it in userspace,
> then it should be done in userspace (or at least there should be an
> option to use userspace -- each use case can choose what's best for
> itself).
>
> [1] https://www.linux-kvm.org/images/3/3d/01x02-Steve_Rutherford-Performant_Security_Hardening_of_KVM.pdf
>

We are already forwarding a few of the category #2 and all
of category #4 SBI extensions to KVM user space which are
not in critical or hot-path.

Majority of SBI extensions in categories #1, #2, #3, and #5
provide critical per-VCPU functionality and many of these
are also in hotpath (such as #1, #3, and #5) hence
implemented in kernel space.

Further, KVM user space lacks required functionality (CSRs,
instructions, or ioctls) to implement many critical SBI extensions
in user space so blanket forwarding of all SBI extensions to
KVM user space is not going to fly.
(Note: Previously, I have already provided many examples)

In short, a hybrid approach (current implementation) is the
best thing where only non-critical and non-hotpath SBI
extensions (few of them) are forwarded to KVM user space
while critical / hot-path SBI extensions (majority of them)
are in kernel space.

Regards,
Anup

Powered by blists - more mailing lists