linux-kernel - Re: [RfC PATCH] drm: fourcc byteorder: brings header file comments in line with reality.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170410161214.305f5daf@eldfell>
Date:   Mon, 10 Apr 2017 16:12:14 +0300
From:   Pekka Paalanen <ppaalanen@...il.com>
To:     Gerd Hoffmann <kraxel@...hat.com>
Cc:     dri-devel@...ts.freedesktop.org,
        open list <linux-kernel@...r.kernel.org>,
        amd-gfx@...ts.freedesktop.org,
        Daniel Vetter <daniel.vetter@...el.com>
Subject: Re: [RfC PATCH] drm: fourcc byteorder: brings header file comments
 in line with reality.

On Mon, 10 Apr 2017 12:12:01 +0200
Gerd Hoffmann <kraxel@...hat.com> wrote:

> Ok, this is really a kickstart for a discussion.  While working on
> graphics support for virtual machines on ppc64 (which exists in both
> little and big endian variants) I've figured the comments in the header
> file don't match reality.  They are not considered little endian (as
> suggested by the comments) but in practice are used as native endian.
> 
> So, go update the comments.
> 
> This patch switches all 32bpp / 8 bpc formats over to native endian.
> Those used/supported by ppc64 virtual machines (virtio-gpu/bochs-drm
> drivers).
> 
> Given that DRM_FORMAT_BIG_ENDIAN isn't used anywhere in the codebase
> I suspect this problem applies to more formats.  When looking at
> drm_mode_legacy_fb_format it seems *all* RGB formats are actually
> native endian not little endian.
> 
> Dunno where we stand in terms of YCbCr.
> 
> Cc: Ville Syrjälä <ville.syrjala@...ux.intel.com>
> Cc: Daniel Vetter <daniel.vetter@...el.com>
> Cc: amd-gfx@...ts.freedesktop.org
> Signed-off-by: Gerd Hoffmann <kraxel@...hat.com>
> ---
>  include/uapi/drm/drm_fourcc.h | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)

Hi,

which software have you used as representative of "reality"?

I worry is that there is potential to have endianess bugs in every
layer of the graphics stack: driver, Mesa, display servers,
compositors, toolkits, and applications. What's worse, two wrongs might
sometimes make a right - incorrectly swapping twice gives you what you
expected in the first place.

We just had a detailed review to get a new pixel format
definitions list in Weston right for LE vs. BE and Pixman vs. DRM vs.
OpenGL. We took the DRM definition literally to be always described in
LE regardless of machine endianess:
https://cgit.freedesktop.org/wayland/weston/commit/?id=903721a6215f474787b5daf02761fbcb1d3a0bb5

We also have this fun problem in Wayland:
https://lists.freedesktop.org/archives/wayland-devel/2017-March/033492.html

To solve that problem, we would like to know if anything existing would
break for each possible solution, but no developers using BE have really
turned up.

We have known for a long time that at least Weston is likely broken on
BE wrt. pixel formats.

We also have a bug for Mesa EGL/Wayland:
https://bugs.freedesktop.org/show_bug.cgi?id=99638

> diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
> index 995c8f9..a7fc81d 100644
> --- a/include/uapi/drm/drm_fourcc.h
> +++ b/include/uapi/drm/drm_fourcc.h
> @@ -85,15 +85,15 @@ extern "C" {
>  #define DRM_FORMAT_BGR888	fourcc_code('B', 'G', '2', '4') /* [23:0] B:G:R little endian */
>  
>  /* 32 bpp RGB */
> -#define DRM_FORMAT_XRGB8888	fourcc_code('X', 'R', '2', '4') /* [31:0] x:R:G:B 8:8:8:8 little endian */
> -#define DRM_FORMAT_XBGR8888	fourcc_code('X', 'B', '2', '4') /* [31:0] x:B:G:R 8:8:8:8 little endian */
> -#define DRM_FORMAT_RGBX8888	fourcc_code('R', 'X', '2', '4') /* [31:0] R:G:B:x 8:8:8:8 little endian */
> -#define DRM_FORMAT_BGRX8888	fourcc_code('B', 'X', '2', '4') /* [31:0] B:G:R:x 8:8:8:8 little endian */
> +#define DRM_FORMAT_XRGB8888	fourcc_code('X', 'R', '2', '4') /* [31:0] x:R:G:B 8:8:8:8 native endian */
> +#define DRM_FORMAT_XBGR8888	fourcc_code('X', 'B', '2', '4') /* [31:0] x:B:G:R 8:8:8:8 native endian */
> +#define DRM_FORMAT_RGBX8888	fourcc_code('R', 'X', '2', '4') /* [31:0] R:G:B:x 8:8:8:8 native endian */
> +#define DRM_FORMAT_BGRX8888	fourcc_code('B', 'X', '2', '4') /* [31:0] B:G:R:x 8:8:8:8 native endian */
>  
> -#define DRM_FORMAT_ARGB8888	fourcc_code('A', 'R', '2', '4') /* [31:0] A:R:G:B 8:8:8:8 little endian */
> -#define DRM_FORMAT_ABGR8888	fourcc_code('A', 'B', '2', '4') /* [31:0] A:B:G:R 8:8:8:8 little endian */
> -#define DRM_FORMAT_RGBA8888	fourcc_code('R', 'A', '2', '4') /* [31:0] R:G:B:A 8:8:8:8 little endian */
> -#define DRM_FORMAT_BGRA8888	fourcc_code('B', 'A', '2', '4') /* [31:0] B:G:R:A 8:8:8:8 little endian */
> +#define DRM_FORMAT_ARGB8888	fourcc_code('A', 'R', '2', '4') /* [31:0] A:R:G:B 8:8:8:8 native endian */
> +#define DRM_FORMAT_ABGR8888	fourcc_code('A', 'B', '2', '4') /* [31:0] A:B:G:R 8:8:8:8 native endian */
> +#define DRM_FORMAT_RGBA8888	fourcc_code('R', 'A', '2', '4') /* [31:0] R:G:B:A 8:8:8:8 native endian */
> +#define DRM_FORMAT_BGRA8888	fourcc_code('B', 'A', '2', '4') /* [31:0] B:G:R:A 8:8:8:8 native endian */
>  
>  #define DRM_FORMAT_XRGB2101010	fourcc_code('X', 'R', '3', '0') /* [31:0] x:R:G:B 2:10:10:10 little endian */
>  #define DRM_FORMAT_XBGR2101010	fourcc_code('X', 'B', '3', '0') /* [31:0] x:B:G:R 2:10:10:10 little endian */

These format definitions are directly referenced by Wayland protocol
extension for dmabufs and Mesa (wl_drm), and they have been copied into
the Wayland core protocol for wl_shm. Changing their definition would
be... awkward.


Thanks,
pq

Content of type "application/pgp-signature" skipped