linux-kernel - Re: [PATCH] drm: fourcc byteorder: brings header file comments in line with reality.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sat, 22 Apr 2017 13:05:22 +0300
From:   Ville Syrjälä <ville.syrjala@...ux.intel.com>
To:     Gerd Hoffmann <kraxel@...hat.com>
Cc:     Christian König <christian.koenig@....com>,
        Jani Nikula <jani.nikula@...ux.intel.com>,
        David Airlie <airlied@...ux.ie>,
        Michel Dänzer <michel@...nzer.net>,
        open list <linux-kernel@...r.kernel.org>,
        dri-devel@...ts.freedesktop.org,
        Pekka Paalanen <ppaalanen@...il.com>,
        Sean Paul <seanpaul@...omium.org>,
        amd-gfx@...ts.freedesktop.org,
        Alex Deucher <alexdeucher@...il.com>,
        Daniel Vetter <daniel.vetter@...el.com>,
        Ilia Mirkin <imirkin@...m.mit.edu>
Subject: Re: [PATCH] drm: fourcc byteorder: brings header file comments in
 line with reality.

On Fri, Apr 21, 2017 at 06:14:31PM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > > My personal opinion is that formats in drm_fourcc.h should be 
> > > independent of the CPU byte order and the function 
> > > drm_mode_legacy_fb_format() and drivers depending on that incorrect 
> > > assumption be fixed instead.
> > 
> > The problem is this isn't a kernel-internal thing any more.  With the
> > addition of the ADDFB2 ioctl the fourcc codes became part of the
> > kernel/userspace abi ...
> 
> Ok, added some printk's to the ADDFB and ADDFB2 code paths and tested a
> bit.  Apparently pretty much all userspace still uses the ADDFB ioctl.
> xorg (modesetting driver) does.  gnome-shell in wayland mode does.
> Seems the big transition to ADDFB2 didn't happen yet.
> 
> I guess that makes changing drm_mode_legacy_fb_format + drivers a
> reasonable option ...

Yeah, I came to the same conclusion after chatting with some
folks on irc.

So my current idea is that we change any driver that wants to follow the
CPU endianness to declare support for big endian formats if the CPU is
big endian. Presumably these are mostly the virtual GPU drivers.

Additonally we'll make the mapping performed by drm_mode_legacy_fb_format()
driver controlled. That way drivers that got changed to follow CPU
endianness can return a framebuffer that matches CPU endianness. And
drivers that expect the GPU endianness to not depend on the CPU
endianness will keep working as they do now. The downside is that users
of the legacy addfb ioctl will need to magically know which endianness
they will get, but that is apparently already the case. And users of
addfb2 will keep on specifying the endianness explicitly with
DRM_FORMAT_BIG_ENDIAN vs. 0.

Does that sound like a workable solution?

-- 
Ville Syrjälä
Intel OTC