linux-kernel - Re: [PATCH v2] drm/virtio: Add suppport for non-native buffer formats

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMuHMdXtUYJmEharJhBXx7D=fA3mQxg6uMP2=4Qgi==2a+kVQw@mail.gmail.com>
Date:   Sun, 19 Nov 2023 19:33:01 +0100
From:   Geert Uytterhoeven <geert@...ux-m68k.org>
To:     Christian Zigotzky <chzigotzky@...osoft.de>
Cc:     Gerd Hoffmann <kraxel@...hat.com>,
        David Airlie <airlied@...hat.com>,
        Gurchetan Singh <gurchetansingh@...omium.org>,
        Chia-I Wu <olvaffe@...il.com>, Daniel Vetter <daniel@...ll.ch>,
        Thomas Zimmermann <tzimmermann@...e.de>,
        Laurent Vivier <lvivier@...hat.com>,
        Javier Martinez Canillas <javierm@...hat.com>,
        Hamza Mahfooz <hamza.mahfooz@....com>,
        linux-m68k@...ts.linux-m68k.org, dri-devel@...ts.freedesktop.org,
        virtualization@...ts.linux-foundation.org,
        linux-kernel@...r.kernel.org, "R.T.Dickinson" <rtd2@...a.co.nz>,
        mad skateman <madskateman@...il.com>,
        Darren Stevens <darren@...vens-zone.net>,
        Christian Zigotzky <info@...osoft.de>
Subject: Re: [PATCH v2] drm/virtio: Add suppport for non-native buffer formats

Hi Christian,

On Sun, Nov 19, 2023 at 5:28 PM Christian Zigotzky
<chzigotzky@...osoft.de> wrote:
> On 16 November 2023 at 03:44 pm, Gerd Hoffmann wrote:
> > On Thu, Nov 16, 2023 at 02:16:54PM +0100, Geert Uytterhoeven wrote:
> >> When using virtgpu on a big-endian machine, e.g. powerpc QEMU:
> >>
> >>      virtio-pci 0000:00:02.0: [drm] *ERROR* fbdev: Failed to setup generic emulation (ret=-2)
> >>
> >> or m68k/virt:
> >>
> >>      virtio-mmio virtio-mmio.125: [drm] *ERROR* fbdev: Failed to setup generic emulation (ret=-2)
> >>
> >> and the graphical display fails to come up.
> >>
> >> Before, the call to drm_mode_addfb() caused a translation from a fourcc
> >> format (XR24) to a bpp/depth pair (32/24) to a potentially different fourcc
> >> format (BX24 on big-endian), due to the quirk processing in
> >> drm_driver_legacy_fb_format().  After, the original fourcc format (XR24)
> >> is passed unmodified.
> >>
> >> However, the virtgpu DRM driver supports only a single format for its
> >> main plane: DRM_FORMAT_HOST_XRGB8888, which is XR24 on little-endian,
> >> and BX24 on big-endian.  I.e. on big-endian, virtgpu does not support
> >> XR24, which is the default DRM format, and must be supported by all
> >> drivers.  Before, this was reported, but didn't lead to a failure:
> >>
> >>      virtio-mmio virtio-mmio.125: [drm] bpp/depth value of 32/24 not supported
> >>      virtio-mmio virtio-mmio.125: [drm] No compatible format found
> >>
> >> As the core virtgpu driver and device support both XR24 and BX24 on both
> >> little-endian and big-endian just fine, fix this extending the list of
> >> supported formats for main plane and cursor plane to XR24/BX24 resp.
> >> AR24/BA24.
> >>
> >> Fixes: 6ae2ff23aa43a0c4 ("drm/client: Convert drm_client_buffer_addfb() to drm_mode_addfb2()")
> >> Reported-by: Christian Zigotzky <chzigotzky@...osoft.de>
> >> Closes: https://lore.kernel.org/r/c47fba21-3ae9-4021-9f4a-09c2670ebdbc@xenosoft.de
> >> Suggested-by: Gerd Hoffmann <kraxel@...hat.com>
> >> Signed-off-by: Geert Uytterhoeven <geert@...ux-m68k.org>
> > Reviewed-by: Gerd Hoffmann <kraxel@...hat.com>
>
> The new patch works but I don't see the virtio-mouse-pci pointer
> anymore. I see the pointer with -device usb-tablet. Please check the
> second patch. I will use the first patch for the RC2 of kernel 6.7.

That's probably partially explained by commit 99748ab64fcc8578 ("drm:
virtio: fix virtio_gpu_cursor_formats"), which forced BA24 for the
cursor plane on big-endian, but unfortunately linked thread doesn't
contain the full picture.

Where is the default cursor format defined?
I see virtio_gpu_mode_dumb_create() still defaults to
DRM_FORMAT_HOST_XRGB8888.  However, that can't be the cause, as the
cursor formats require an alpha channel.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@...ux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds