Message-ID:
 <SN6PR02MB4157ECDBBC89D0E838402528D4102@SN6PR02MB4157.namprd02.prod.outlook.com>
Date: Mon, 6 Jan 2025 19:49:15 +0000
From: Michael Kelley <mhklinux@...look.com>
To: Stanislav Kinsburskii <skinsburskii@...ux.microsoft.com>
CC: Roman Kisel <romank@...ux.microsoft.com>, "hpa@...or.com" <hpa@...or.com>,
	"kys@...rosoft.com" <kys@...rosoft.com>, "bp@...en8.de" <bp@...en8.de>,
	"dave.hansen@...ux.intel.com" <dave.hansen@...ux.intel.com>,
	"decui@...rosoft.com" <decui@...rosoft.com>, "eahariha@...ux.microsoft.com"
	<eahariha@...ux.microsoft.com>, "haiyangz@...rosoft.com"
	<haiyangz@...rosoft.com>, "mingo@...hat.com" <mingo@...hat.com>,
	"nunodasneves@...ux.microsoft.com" <nunodasneves@...ux.microsoft.com>,
	"tglx@...utronix.de" <tglx@...utronix.de>, "tiala@...rosoft.com"
	<tiala@...rosoft.com>, "wei.liu@...nel.org" <wei.liu@...nel.org>,
	"linux-hyperv@...r.kernel.org" <linux-hyperv@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"x86@...nel.org" <x86@...nel.org>, "apais@...rosoft.com"
	<apais@...rosoft.com>, "benhill@...rosoft.com" <benhill@...rosoft.com>,
	"ssengar@...rosoft.com" <ssengar@...rosoft.com>, "sunilmut@...rosoft.com"
	<sunilmut@...rosoft.com>, "vdso@...bites.dev" <vdso@...bites.dev>
Subject: RE: [PATCH v5 3/5] hyperv: Enable the hypercall output page for the
 VTL mode

From: Stanislav Kinsburskii <skinsburskii@...ux.microsoft.com> Sent: Monday, January 6, 2025 11:19 AM
> 
> On Mon, Jan 06, 2025 at 06:18:51PM +0000, Michael Kelley wrote:
> > From: Stanislav Kinsburskii <skinsburskii@...ux.microsoft.com> Sent: Monday, January 6, 2025 9:23 AM
> > >
> > > On Fri, Jan 03, 2025 at 10:08:05PM +0000, Michael Kelley wrote:
> > > > From: Stanislav Kinsburskii <skinsburskii@...ux.microsoft.com> Sent: Friday, January
> > > 3, 2025 11:20 AM
> > > > >
> > > > > On Mon, Dec 30, 2024 at 10:09:39AM -0800, Roman Kisel wrote:
> > > > > > Due to the hypercall page not being allocated in the VTL mode,
> > > > > > the code resorts to using a part of the input page.
> > > > > >
> > > > > > Allocate the hypercall output page in the VTL mode thus enabling
> > > > > > it to use it for output and share code with dom0.
> > > > > >
> > > > > > Signed-off-by: Roman Kisel <romank@...ux.microsoft.com>
> > > > > > ---
> > > > > >  drivers/hv/hv_common.c | 6 +++---
> > > > > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c
> > > > > > index c6ed3ba4bf61..c983cfd4d6c0 100644
> > > > > > --- a/drivers/hv/hv_common.c
> > > > > > +++ b/drivers/hv/hv_common.c
> > > > > > @@ -340,7 +340,7 @@ int __init hv_common_init(void)
> > > > > >  	BUG_ON(!hyperv_pcpu_input_arg);
> > > > > >
> > > > > >  	/* Allocate the per-CPU state for output arg for root */
> > > > > > -	if (hv_root_partition) {
> > > > > > +	if (hv_root_partition || IS_ENABLED(CONFIG_HYPERV_VTL_MODE)) {
> > > > >
> > > > > This check doesn't look nice.
> > > > > First of all, IS_ENABLED(CONFIG_HYPERV_VTL_MODE) doesn't mean that this
> > > > > particular kernel is being booted in VTL other that VTL0.
> > > >
> > > > Actually, it does mean that. Kernels built with CONFIG_HYPERV_VTL_MODE=y
> > > > will not run as a normal guest in VTL 0. See the third paragraph of the
> > > > "help" section for HYPERV_VTL_MODE in drivers/hv/Kconfig.
> > > >
> > >
> > > Thanks for pointing to this piece.
> > >
> > > This limitation looks aritificial to me and as VTL support in Linux is
> > > currently being extended beyond Underhill support, keeping this
> > > restriction makes some further development in scope of LVBS support
> > > complicated and error prone due to potential ABI mismatches between
> > > Linux kernels in different VTLs.
> > >
> > > IOW, making the same kernel properly bootable (or - worse - explicitly
> > > un-bootable) in different VTLs is a more robust way in the long run.
> >
> > The reason for the limitation is the sequencing of early Hyper-V-related
> > initialization steps. Knowing at runtime whether you are running at
> > VTL0 or some other VTL requires making a hypercall in get_vtl().
> > Unfortunately, the machinery for making a hypercall (setting the guest
> > OS ID, and allocating the x86 hypercall page) is established relatively late
> > during initialization, in hyperv_init(). But running in other than VTL0
> > requires the initializations that are done in hv_vtl_init_platform(), which
> > must be done much earlier. There's no clear way out of this conundrum
> > purely on the Linux guest side.
> >
> > To solve the conundrum on x86, one possibility to consider is having
> > Hyper-V make HV_REGISTER_VSM_VP_STATUS available as a synthetic
> > MSR, which can be read without making a hypercall. This register could be
> > read in ms_hyperv_init_platform() to know if running at VTL0 or not.
> > Using synthetic MSRs is how other aspects of early Hyper-V-related
> > initialization is done in Linux on x86.
> >
> > I think there's some discussion on the x86 sequencing issues on LKML
> > from when the VTL code was first added. I was part of that discussion, but
> > don't remember all the details. There might additional issues raised in
> > that discussion.
> >
> > The sequencing issues would also need to be sorted out on the arm64
> > side, as they are different from x86. We don't have an early Hyper-V
> > specific hook like ms_hyperv_init_platform() on the arm64 side, so
> > that might be problem. But on the flip side, we also don't have the
> > x86-specific messiness that hv_vtl_init_platform() handles. Also,
> > there are no synthetic MSRs on arm64, so register accesses always
> > use hypercalls, but there's no hypercall page needed. On balance, I
> > think getting VTL stuff initialized on arm64 will be easier, but I'm not sure.
> >
> > Michael
> 
> Thank you for summarizing this up. This aligns with my understanding.
> 
> Since VTL1 firmware is a payload for VTL0 kernel, one of my proposals to
> the original message was to explicitly notify the kernel it's running in
> VTL1 via command line argument and/or DT.
> 

OK, yes.  Rather than non-VTL0 behavior being a kernel build-time decision,
make it a kernel boot-line based decision. A runtime decision based on
detecting the VTL is hard as described above.

I don't immediately see an initialization sequence problem in using a kernel
boot line parameter to specify running in non-VTL0. I'm unsure if DT would
be available at the time ms_hyperv_init_platform() runs and decides to call
hv_vtl_init_platform(). Have you looked to see?

Michael

Powered by blists - more mailing lists