linux-kernel - Re: [PATCH 2/2] drm: virtio: fix virtio_gpu_mode_dumb

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Mon, 03 Apr 2017 11:02:15 +0200
From:   Gerd Hoffmann <kraxel@...hat.com>
To:     Daniel Vetter <daniel@...ll.ch>
Cc:     dri-devel@...ts.freedesktop.org, lvivier@...hat.com,
        open list <linux-kernel@...r.kernel.org>,
        "open list:VIRTIO GPU DRIVER" 
        <virtualization@...ts.linux-foundation.org>
Subject: Re: [PATCH 2/2] drm: virtio: fix virtio_gpu_mode_dumb_create

On Mo, 2017-04-03 at 09:53 +0200, Daniel Vetter wrote:
> On Mon, Apr 03, 2017 at 09:08:45AM +0200, Gerd Hoffmann wrote:
> > Lookup format using virtio_gpu_translate_format()
> > instead of hardcoding it.  Fixes xorg display on
> > bigendian guests (i.e. ppc64).
> > 
> > Signed-off-by: Gerd Hoffmann <kraxel@...hat.com>
> 
> ow ... :(
> 
> With what kind of client have you tested this? fbdev? kms native?

text console (fbcon) is fine, xorg (modesetting driver) fails.

> Should we patch drm_fourcc.h and essentially state that _all_ drm_fourcc
> are in native endian, don't dare mixing things up?

Well, I don't think this needs clarification.  I never assumed that
fourcc is something else than native endian.  It really was just an
oversight in the virtio driver.

One code path (which creates the framebuffer for fbdev emulation) did
proper drm_fourcc -> virtio format mapping, including endianness
handling.  Patch 1/2 of this series factors this code out into a helper
function so we can easily reuse it.

The other code path (for creating dumb buffers) had the virtio format
hardcoded which happened to work correctly in the little endian case.

cheers,
  Gerd