linux-kernel - Re: [PATCH] drm: fourcc byteorder: brings header file comments in line with reality.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1492898365.25675.77.camel@redhat.com>
Date:   Sat, 22 Apr 2017 23:59:25 +0200
From:   Gerd Hoffmann <kraxel@...hat.com>
To:     Ville Syrjälä <ville.syrjala@...ux.intel.com>
Cc:     Christian König <christian.koenig@....com>,
        Jani Nikula <jani.nikula@...ux.intel.com>,
        David Airlie <airlied@...ux.ie>,
        Michel Dänzer <michel@...nzer.net>,
        open list <linux-kernel@...r.kernel.org>,
        dri-devel@...ts.freedesktop.org,
        Pekka Paalanen <ppaalanen@...il.com>,
        Sean Paul <seanpaul@...omium.org>,
        amd-gfx@...ts.freedesktop.org,
        Alex Deucher <alexdeucher@...il.com>,
        Daniel Vetter <daniel.vetter@...el.com>,
        Ilia Mirkin <imirkin@...m.mit.edu>
Subject: Re: [PATCH] drm: fourcc byteorder: brings header file comments in
 line with reality.

  Hi,

> > I guess that makes changing drm_mode_legacy_fb_format + drivers a
> > reasonable option ...
> 
> Yeah, I came to the same conclusion after chatting with some
> folks on irc.
> 
> So my current idea is that we change any driver that wants to follow the
> CPU endianness to declare support for big endian formats if the CPU is
> big endian. Presumably these are mostly the virtual GPU drivers.

Agree.  Easy.

> Additonally we'll make the mapping performed by drm_mode_legacy_fb_format()
> driver controlled.

I don't think this is useful.  IMO drm_mode_legacy_fb_format should
return host endian formats unconditionally.

> That way drivers that got changed to follow CPU
> endianness can return a framebuffer that matches CPU endianness. And
> drivers that expect the GPU endianness to not depend on the CPU
> endianness will keep working as they do now. The downside is that users
> of the legacy addfb ioctl will need to magically know which endianness
> they will get, but that is apparently already the case.

Existing userspace expects host endian, and IMO we should maintain that
behavior.

> And users of
> addfb2 will keep on specifying the endianness explicitly with
> DRM_FORMAT_BIG_ENDIAN vs. 0.

I'd drop DRM_FORMAT_BIG_ENDIAN.

At least for the virt drivers it doesn't buy us anything.  They support
32bpp / 8 bpc formats only[1], and for those I can specify the
byteswapped format version without a bigendian flag because we have
fourccs for everything we need.


There is a WIP patch series at
https://www.kraxel.org/cgit/linux/log/?h=drm-byteorder

Needs more testing and better commit messages.  /me plans to polish &
post next week, but feel free to look and comment.

cheers,
  Gerd

[1] Everything else is a PITA to deal with on the host side because
    I can't offload that to pixman.  There is no support for
    PIXMAN_r5g6b5 or PIXMAN_x2b10g10r10 in non-host byte order.