linux-kernel - Re: [PATCH 1/3] drm: fourcc byteorder: drop DRM_FORMAT_BIG

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1493803481.8581.163.camel@redhat.com>
Date:   Wed, 03 May 2017 11:24:41 +0200
From:   Gerd Hoffmann <kraxel@...hat.com>
To:     Michel Dänzer <michel@...nzer.net>
Cc:     Pekka Paalanen <ppaalanen@...il.com>,
        amd-gfx mailing list <amd-gfx@...ts.freedesktop.org>,
        Emil Velikov <emil.l.velikov@...il.com>,
        open list <linux-kernel@...r.kernel.org>,
        ML dri-devel <dri-devel@...ts.freedesktop.org>,
        Daniel Vetter <daniel.vetter@...el.com>
Subject: Re: [PATCH 1/3] drm: fourcc byteorder: drop DRM_FORMAT_BIG_ENDIAN

  Hi,

> > R600+ supports bigendian framebuffer formats, so no byteswapping on
> > access is needed.  Not sure whenever that includes 16bpp formats or
> > whenever this is limited to the 8 bit-per-color formats [...]
> 
> It includes 16bpp. Looking at
> drivers/gpu/drm/radeon/atombios_crtc.c:dce4_crtc_do_set_base(), it sets
> up byte-swapping for all multi-byte formats, so it effectively treats
> all those formats as if they had DRM_FORMAT_BIG_ENDIAN set.

> If the radeon (and amdgpu) driver were to be changed to use
> drm_mode_legacy_fb_format_he for >= R600, that must also handle 16 bpp,
> which requires DRM_FORMAT_BIG_ENDIAN. So I still don't see how that can
> be removed or even deprecated.

Ok.

Dropped patch #1.

Updated patch #2 to include all formats returned by
drm_mode_legacy_fb_format, and also renamed them to DRM_FORMAT_HOST_*.

Question is how to go forward with patch #3.  I'd prefer to not add
drm_mode_legacy_fb_format_he if possible.  Is there a chance to adapt
the radeon and nvidia drivers to a fixed drm_mode_legacy_fb_format
function (returning be formats on be) without invasive changes?  Given
they both treat formats as if they had DRM_FORMAT_BIG_ENDIAN set this
could (with the help of the extended patch #2) be a simple
s/DRM_FORMAT_/DRM_FORMAT_HOST/ at the right places ...

Michael?  Ilia?

cheers,
  Gerd