linux-kernel - Re: [PATCH 3/6] drm: fourcc byteorder: add bigendian support to drm_mode_legacy_fb

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <1493373758.23357.27.camel@redhat.com>
Date:   Fri, 28 Apr 2017 12:02:38 +0200
From:   Gerd Hoffmann <kraxel@...hat.com>
To:     Michel Dänzer <michel@...nzer.net>
Cc:     Daniel Vetter <daniel.vetter@...el.com>,
        amd-gfx@...ts.freedesktop.org,
        open list <linux-kernel@...r.kernel.org>,
        dri-devel@...ts.freedesktop.org
Subject: Re: [PATCH 3/6] drm: fourcc byteorder: add bigendian support to
 drm_mode_legacy_fb_format

  Hi,

> > So just not using the swapping indeed looks like the only sensible
> > option.  Which in turn implies there is no BGRA8888 support for dumb
> > bos.  Hmm, I can see the problem.  Userspace expectation appears to be
> > that ADDFB configures a native endian framebuffer, which the driver
> > simply can't do on bigendian.
> 
> ... with pre-R600 GPUs.

Sure.

> > So, what can/should the driver do here?  Throw errors for ADDFB and
> > force userspace to use ADDFB2?  From a design point of view the best
> > option, but in the other hand I suspect that could break the xorg radeon
> > driver ...
> 
> Yes, it would.

> One thing we could do is provide a way for userspace to query the
> effective format(s) as seen by the GPU and/or CPU.

We already have almost no testing on bigendian.  I doubt adding generic
interfaces specifically to handle this case is going to work because
most userspace will simply not implement that correctly (or at all).

Having support for this in the radeon ioctls might work, because only
radeon kernel + xorg driver have to get things right then.  But I
suspect we already have that.  You've mentioned elsewhere in the thread
that the xorg driver doesn't turn on byteswapping, so the ability to
configure that seems to be somewhere in the API ...

> It might also make sense for the radeon driver to set the
> RADEON_TILING_SWAP_{16,32}BIT flags for dumb BOs.

That could work.  But I guess someone with test hardware needs to try,
to make sure we don't miss corner cases here.

cheers,
  Gerd