linux-kernel - Re: [PATCH] drm: fourcc byteorder: brings header file comments in line with reality.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sat, 22 Apr 2017 12:50:56 +0300
From:   Ville Syrjälä <ville.syrjala@...ux.intel.com>
To:     Ilia Mirkin <imirkin@...m.mit.edu>
Cc:     Gerd Hoffmann <kraxel@...hat.com>,
        "dri-devel@...ts.freedesktop.org" <dri-devel@...ts.freedesktop.org>,
        Daniel Vetter <daniel.vetter@...el.com>,
        Pekka Paalanen <ppaalanen@...il.com>,
        Michel Dänzer <michel@...nzer.net>,
        Alex Deucher <alexdeucher@...il.com>,
        amd-gfx@...ts.freedesktop.org,
        Jani Nikula <jani.nikula@...ux.intel.com>,
        Sean Paul <seanpaul@...omium.org>,
        David Airlie <airlied@...ux.ie>,
        open list <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] drm: fourcc byteorder: brings header file comments in
 line with reality.

On Sat, Apr 22, 2017 at 01:07:57AM -0400, Ilia Mirkin wrote:
> On Fri, Apr 21, 2017 at 12:59 PM, Ville Syrjälä
> <ville.syrjala@...ux.intel.com> wrote:
> > On Fri, Apr 21, 2017 at 10:49:49AM -0400, Ilia Mirkin wrote:
> >> On Fri, Apr 21, 2017 at 3:58 AM, Gerd Hoffmann <kraxel@...hat.com> wrote:
> >> > While working on graphics support for virtual machines on ppc64 (which
> >> > exists in both little and big endian variants) I've figured the comments
> >> > for various drm fourcc formats in the header file don't match reality.
> >> >
> >> > Comments says the RGB formats are little endian, but in practice they
> >> > are native endian.  Look at the drm_mode_legacy_fb_format() helper.  It
> >> > maps -- for example -- bpp/depth 32/24 to DRM_FORMAT_XRGB8888, no matter
> >> > whenever the machine is little endian or big endian.  The users of this
> >> > function (fbdev emulation, DRM_IOCTL_MODE_ADDFB) expect the framebuffer
> >> > is native endian, not little endian.  Most userspace also operates on
> >> > native endian only.
> >> >
> >> > So, go update the comments for all 16+24+32 bpp RGB formats.
> >> >
> >> > Leaving the yuv formats as-is.  I have no idea if and how those are used
> >> > on bigendian machines.
> >>
> >> I think this is premature. The current situation is that I can't get
> >> modetest to work *at all* on my NV34 / BE setup (I mean, it runs, just
> >> the colors displayed are wrong). I believe that currently it packs
> >> things in "cpu native endian". I've tried futzing with that without
> >> much success, although I didn't spend too much time on it. I have a
> >> NV34 plugged into my LE setup as well although I haven't tested to
> >> double-check that it all works there. However I'm quite sure it used
> >> to, as I used modetest to help develop the YUV overlay support for
> >> those GPUs.
> >
> > I just took a quick stab at fixing modetest to respect the current
> > wording in drm_fourcc.h:
> >
> > git://github.com/vsyrjala/libdrm.git modetest_endian
> 
> Looks like there was some careless testing on my part :( So ... it
> looks like the current modetest without those changes does, in fact,
> work on NV34/BE. With the changes, it breaks (and the handling of the
> b* modes is a little broken in those patches -- they're not selectable
> from the cmdline.) Which means that, as Michel & co predicted, it
> appears to be taking BE input not LE input. This is very surprising to
> me, but it is what it is. As I mentioned before, the details of how
> the "BE" mode works on the GPUs is largely unknown to us beyond a few
> basics. Note that only XR24 works, AR24 ends up with all black
> displayed. This also happens on LE.

Did you try 8bpp or 16bpp formats? I expect that if you've just blindly
enabled some magic byte swapper in the hardware it will only for
a specific pixel size.

-- 
Ville Syrjälä
Intel OTC