Message-ID:
 <SN6PR02MB4157FD9AB9114C8DD6FF4B82D49FA@SN6PR02MB4157.namprd02.prod.outlook.com>
Date: Tue, 20 May 2025 02:07:48 +0000
From: Michael Kelley <mhklinux@...look.com>
To: Saurabh Singh Sengar <ssengar@...ux.microsoft.com>
CC: Saurabh Singh Sengar <ssengar@...rosoft.com>, KY Srinivasan
	<kys@...rosoft.com>, Haiyang Zhang <haiyangz@...rosoft.com>,
	"wei.liu@...nel.org" <wei.liu@...nel.org>, Dexuan Cui <decui@...rosoft.com>,
	"deller@....de" <deller@....de>, "javierm@...hat.com" <javierm@...hat.com>,
	"arnd@...db.de" <arnd@...db.de>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "linux-hyperv@...r.kernel.org"
	<linux-hyperv@...r.kernel.org>, "stable@...r.kernel.org"
	<stable@...r.kernel.org>
Subject: RE: [EXTERNAL] [PATCH 1/1] Drivers: hv: Always select CONFIG_SYSFB
 for Hyper-V guests

From: Saurabh Singh Sengar <ssengar@...ux.microsoft.com> Sent: Monday, May 19, 2025 9:55 AM
> 
> On Sat, May 17, 2025 at 06:47:22PM +0000, Michael Kelley wrote:
> > From: Saurabh Singh Sengar <ssengar@...ux.microsoft.com> Sent: Saturday, May 17, 2025 9:14 AM
> > >
> > > On Sat, May 17, 2025 at 01:34:20PM +0000, Michael Kelley wrote:
> > > > From: Saurabh Singh Sengar <ssengar@...rosoft.com> Sent: Friday, May 16, 2025 9:38 PM
> > > > >
> > > > > > From: Michael Kelley <mhklinux@...look.com>
> > > > > >
> > > > > > The Hyper-V host provides guest VMs with a range of MMIO addresses that
> > > > > > guest VMBus drivers can use. The VMBus driver in Linux manages that MMIO
> > > > > > space, and allocates portions to drivers upon request. As part of managing
> > > > > > that MMIO space in a Generation 2 VM, the VMBus driver must reserve the
> > > > > > portion of the MMIO space that Hyper-V has designated for the synthetic
> > > > > > frame buffer, and not allocate this space to VMBus drivers other than graphics
> > > > > > framebuffer drivers. The synthetic frame buffer MMIO area is described by
> > > > > > the screen_info data structure that is passed to the Linux kernel at boot time,
> > > > > > so the VMBus driver must access screen_info for Generation 2 VMs. (In
> > > > > > Generation 1 VMs, the framebuffer MMIO space is communicated to the
> > > > > > guest via a PCI pseudo-device, and access to screen_info is not needed.)
> > > > > >
> > > > > > In commit a07b50d80ab6 ("hyperv: avoid dependency on screen_info") the
> > > > > > VMBus driver's access to screen_info is restricted to when CONFIG_SYSFB is
> > > > > > enabled. CONFIG_SYSFB is typically enabled in kernels built for Hyper-V by
> > > > > > virtue of having at least one of CONFIG_FB_EFI, CONFIG_FB_VESA, or
> > > > > > CONFIG_SYSFB_SIMPLEFB enabled, so the restriction doesn't usually affect
> > > > > > anything. But it's valid to have none of these enabled, in which case
> > > > > > CONFIG_SYSFB is not enabled, and the VMBus driver is unable to properly
> > > > > > reserve the framebuffer MMIO space for graphics framebuffer drivers. The
> > > > > > framebuffer MMIO space may be assigned to some other VMBus driver, with
> > > > > > undefined results. As an example, if a VM is using a PCI pass-thru NVMe
> > > > > > controller to host the OS disk, the PCI NVMe controller is probed before any
> > > > > > graphic devices, and the NVMe controller is assigned a portion of the
> > > > > > framebuffer MMIO space.
> > > > > > Hyper-V reports an error to Linux during the probe, and the OS disk fails to
> > > > > > get setup. Then Linux fails to boot in the VM.
> > > > > >
> > > > > > Fix this by having CONFIG_HYPERV always select SYSFB. Then the VMBus
> > > > > > driver in a Gen 2 VM can always reserve the MMIO space for the graphics
> > > > > > framebuffer driver, and prevent the undefined behavior.
> > > > >
> > > > > One question: Shouldn't the SYSFB be selected by actual graphics framebuffer driver
> > > > > which is expected to use it. With this patch this option will be enabled irrespective
> > > > > if there is any user for it or not, wondering if we can better optimize it for such systems.
> > > > >
> > > >
> > > > That approach doesn't work. For a cloud-based server, it might make
> > > > sense to build a kernel image without either of the Hyper-V graphics
> > > > framebuffer drivers (DRM_HYPERV or HYPERV_FB) since in that case the
> > > > Linux console is the serial console. But the problem could still occur
> > > > where a PCI pass-thru NVMe controller tries to use the MMIO space
> > > > that Hyper-V intends for the framebuffer. That problem is directly tied
> > > > to CONFIG_SYSFB because it's the VMBus driver that must treat the
> > > > framebuffer MMIO space as special. The absence or presence of a
> > > > framebuffer driver isn't the key factor, though we've been (incorrectly)
> > > > relying on the presence of a framebuffer driver to set CONFIG_SYSFB.
> > > >
> > >
> > > Thank you for the clarification. I was concerned because SYSFB is not currently
> > > enabled in the OpenHCL kernel, and our goal is to keep the OpenHCL configuration
> > > as minimal as possible. I haven't yet looked into the details to determine
> > > whether this might have any impact on the kernel binary size or runtime memory
> > > usage. I trust this won't affect negatively.
> > >
> > > OpenHCL Config Ref:
> > > https://github.com/microsoft/OHCL-Linux-Kernel/blob/product/hcl-main/6.12/Microsoft/hcl-x64.config
> > >
> >
> > Good point.
> >
> > The OpenHCL code tree has commit a07b50d80ab6 that restricts the
> > screen_info to being available only when CONFIG_SYSFB is enabled.
> > But since OpenHCL in VTL2 gets its firmware info via OF instead of ACPI,
> > I'm unsure what the Hyper-V host tells it about available MMIO space,
> > and whether that space includes MMIO space for a framebuffer. If it
> > doesn't, then OpenHCL won't have the problem I describe above, and
> > it won't need CONFIG_SYSFB. This patch could be modified to do
> >
> > select SYSFB if !HYPERV_VTL_MODE
> 
> I am worried that this is not very scalable, there could be more such
> Hyper-V systems in future.

I could see scalability being a problem if there were 20 more such
Hyper-V systems in the future. But if there are just 2 or 3 more, that
seems like it would be manageable.

Regardless, I'm OK with doing this with or without the
"if !HYPERV_VTL_MODE". I don't think we should just drop this
entirely. When playing around with various framebuffers drivers
a few weeks back, I personally encountered the problem of having
built a kernel that wouldn't boot in an Azure VM with an NVMe OS
disk. I couldn't figure out why probing the NVMe controller failed.
It took me an hour to sort out what was happening, and I was
familiar with the Hyper-V PCI driver. I'd like to prevent such a
problem from happening to someone else.

> 
> >
> > Can you find out what MMIO space Hyper-V provides to VTL2 via OF?
> > It would make sense if no framebuffer is provided. And maybe
> > screen_info itself is not set up when VTL2 is loaded, which would
> > also make adding CONFIG_SYSFB pointless for VTL2.
> 
> I can only see below address range passed for MMIO to VMBus driver:
> ranges = <0x0f 0xf0000000 0x0f 0xf0000000 0x10000000>;

I'm guessing the above text is what shows up in DT?  I'm not sure
how to interpret it. In normal guests, Hyper-V offers a "low MMIO"
range that is below the 4 GiB line, and a "high MMIO" range that
is just before the 64 GiB line. In a normal guest in Azure, I see the
MLX driver using 0xfc0000000, which would be just below the 64 GiB
line, and in the "high MMIO" range. The "0x0f 0xf0000000" in DT might
be physical address 0xff0000000, which is consistent with the
"high MMIO" range.  I'm not sure how to interpret the second
occurrence of "0xf 0xf0000000".  I'm guessing the 0x10000000
(256 Mib) is the length of the available range, which would also
make sense.

The framebuffer address is always in the "low MMIO" range. So
if my interpretation is anywhere close to correct, DT isn't
specifying any MMIO space for a framebuffer, and there's
no need for CONFIG_SYSFB in a kernel running in VTL2.

What's your preference for how to proceed? Adding CONFIG_SYSFB
probably *will* increase the kernel code size, but I don't know
by how much. I can do a measurement.

Michael

> 
> I don't think we have any use of scrren_info or framebuffer in OpenHCL.
> 
> - Saurabh

Powered by blists - more mailing lists