linux-kernel - Re: [PATCH 3/6] drm: fourcc byteorder: add bigendian support to drm_mode_legacy_fb

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 26 Apr 2017 07:53:10 +0200
From:   Gerd Hoffmann <kraxel@...hat.com>
To:     Michel Dänzer <michel@...nzer.net>
Cc:     dri-devel@...ts.freedesktop.org,
        open list <linux-kernel@...r.kernel.org>,
        amd-gfx@...ts.freedesktop.org,
        Daniel Vetter <daniel.vetter@...el.com>
Subject: Re: [PATCH 3/6] drm: fourcc byteorder: add bigendian support to
 drm_mode_legacy_fb_format

On Di, 2017-04-25 at 12:18 +0900, Michel Dänzer wrote:
> On 24/04/17 03:25 PM, Gerd Hoffmann wrote:
> > Return correct fourcc codes on bigendian.  Drivers must be adapted to
> > this change.
> > 
> > Signed-off-by: Gerd Hoffmann <kraxel@...hat.com>
> 
> Just to reiterate, this won't work for the radeon driver, which programs
> the GPU to use (effectively, per the current definition that these are
> little endian GPU formats) DRM_FORMAT_XRGB8888 with pre-R600 and
> DRM_FORMAT_BGRX8888 with >= R600.

Hmm, ok, how does bigendian fbdev emulation work on pre-R600 then?

> > +#ifdef __BIG_ENDIAN
> > +	switch (bpp) {
> > +	case 8:
> > +		fmt = DRM_FORMAT_C8;
> > +		break;
> > +	case 24:
> > +		fmt = DRM_FORMAT_BGR888;
> > +		break;
> 
> BTW, endianness as a concept cannot apply to 8 or 24 bpp formats.

I could move the 8 bpp case out of the #ifdef somehow, but code
readability will suffer then I think ...

For 24 we have different byte orderings, but yes, you can't switch from
one to the other with byteswapping.  Probably one of the reasons why
this format is pretty much out of fashion these days ...

cheers,
  Gerd