Message-ID:
 <SN6PR02MB415740F004283A40C3C41490D4D82@SN6PR02MB4157.namprd02.prod.outlook.com>
Date: Thu, 20 Mar 2025 20:15:50 +0000
From: Michael Kelley <mhklinux@...look.com>
To: Geert Uytterhoeven <geert@...ux-m68k.org>
CC: Helge Deller <deller@....de>, "linux-fbdev@...r.kernel.org"
	<linux-fbdev@...r.kernel.org>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "linux-hyperv@...r.kernel.org"
	<linux-hyperv@...r.kernel.org>
Subject: RE: fbdev deferred I/O broken in some scenarios

From: Geert Uytterhoeven <geert@...ux-m68k.org> Sent: Thursday, March 20, 2025 3:46 AM
> 
> Hi Michael,
> 
> On Wed, 19 Mar 2025 at 21:29, Michael Kelley <mhklinux@...look.com> wrote:
> > From: Helge Deller <deller@....de> Sent: Tuesday, March 18, 2025 1:16 AM
> > > On 3/18/25 03:05, Michael Kelley wrote:
> > > > I've been trying to get mmap() working with the hyperv_fb.c fbdev driver, which
> > > > is for Linux guests running on Microsoft's Hyper-V hypervisor. The hyperv_fb driver
> > > > uses fbdev deferred I/O for performance reasons. But it looks to me like fbdev
> > > > deferred I/O is fundamentally broken when the underlying framebuffer memory
> > > > is allocated from kernel memory (alloc_pages or dma_alloc_coherent).
> > > >
> > > > The hyperv_fb.c driver may allocate the framebuffer memory in several ways,
> > > > depending on the size of the framebuffer specified by the Hyper-V host and the VM
> > > > "Generation".  For a Generation 2 VM, the framebuffer memory is allocated by the
> > > > Hyper-V host and is assigned to guest MMIO space. The hyperv_fb driver does a
> > > > vmalloc() allocation for deferred I/O to work against. This combination handles mmap()
> > > > of /dev/fb<n> correctly and the performance benefits of deferred I/O are substantial.
> > > >
> > > > But for a Generation 1 VM, the hyperv_fb driver allocates the framebuffer memory in
> > > > contiguous guest physical memory using alloc_pages() or dma_alloc_coherent(), and
> > > > informs the Hyper-V host of the location. In this case, mmap() with deferred I/O does
> > > > not work. The mmap() succeeds, and user space updates to the mmap'ed memory are
> > > > correctly reflected to the framebuffer. But when the user space program does munmap()
> > > > or terminates, the Linux kernel free lists become scrambled and the kernel eventually
> > > > panics. The problem is that when munmap() is done, the PTEs in the VMA are cleaned
> > > > up, and the corresponding struct page refcounts are decremented. If the refcount goes
> > > > to zero (which it typically will), the page is immediately freed. In this way, some or all
> > > > of the framebuffer memory gets erroneously freed. From what I see, the VMA should
> > > > be marked VM_PFNMAP when allocated memory kernel is being used as the
> > > > framebuffer with deferred I/O, but that's not happening. The handling of deferred I/O
> > > > page faults would also need updating to make this work.
> 
> I assume this is triggered by running any fbdev userspace that uses
> mmap(), e.g. fbtest?

Yes.  I have my own little program, but it is similar to what I see fbtest does.

> 
> > > > The fbdev deferred I/O support was originally added to the hyperv_fb driver in the
> > > > 5.6 kernel, and based on my recent experiments, it has never worked correctly when
> > > > the framebuffer is allocated from kernel memory. fbdev deferred I/O support for using
> > > > kernel memory as the framebuffer was originally added in commit 37b4837959cb9
> > > > back in 2008 in Linux 2.6.29. But I don't see how it ever worked properly, unless
> > > > changes in generic memory management somehow broke it in the intervening years.
> > > >
> > > > I think I know how to fix all this. But before working on a patch, I wanted to check
> > > > with the fbdev community to see if this might be a known issue and whether there
> > > > is any additional insight someone might offer. Thanks for any comments or help.
> > >
> > > I haven't heard of any major deferred-i/o issues since I've jumped into fbdev
> > > maintenance. But you might be right, as I haven't looked much into it yet and
> > > there are just a few drivers using it.
> >
> > Thanks for the input. In the fbdev directory, there are 9 drivers using deferred I/O.
> > Of those, 6 use vmalloc() to allocate the framebuffer, and that path works just fine.
> > The other 3 use alloc_pages(), dma_alloc_coherent(), or __get_free_pages(), all of
> > which manifest the underlying problem when munmap()'ed.  Those 3 drivers are:
> >
> > * hyperv_fb.c, which I'm working with
> > * sh_mobile_lcdcfb.c
> > * ssd1307fb.c
> 
> Nowadays sh_mobile_lcdcfb is used only on various SuperH boards
> (I have no hardware to test).
> 
> sh_mobile_lcdcfb was used on ARM-based SH/R-Mobile SoCs until DT
> support was added to the DRM driver for the corresponding hardware.
> The platform using it was migrated to DRM in commit 138588e9fa237f97
> ("ARM: dts: renesas: r8a7740: Add LCDC nodes") in v6.8). At the time
> of the conversion, fbtest worked fine with sh_mobile_lcdcfb.

OK, good to know. sh_mobile_lcdcfb gets its framebuffer using
dma_alloc_coherent(). Do you recall how big the framebuffer was?
If over 4 MiB, dma_alloc_coherent() would have allocated from CMA,
and that works OK.

> 
> Deferred I/O is also used in DRM drivers for displays that are connected
> using I2C or SPI.  Last time I tried the st7735r driver, it worked fine
> with fbtest.  That was also on arm32, though.

The st7735r driver appears to use fbtft-core.c, and that does a vmalloc()
for the screen buffer, so it won't have the problem I'm seeing.

Thanks for the help ....

Michael

Powered by blists - more mailing lists