linux-kernel - Re: CTM and other color related properties

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20170329091420.GR30290@intel.com>
Date:   Wed, 29 Mar 2017 12:14:20 +0300
From:   Ville Syrjälä <ville.syrjala@...ux.intel.com>
To:     Jyri Sarha <jsarha@...com>
Cc:     "dri-devel@...ts.freedesktop.org" <dri-devel@...ts.freedesktop.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        jani.nikula@...ux.intel.com, Sean Paul <seanpaul@...omium.org>,
        Lionel G Landwerlin <lionel.g.landwerlin@...el.com>,
        Daniel Vetter <daniel@...ll.ch>,
        "Valkeinen, Tomi" <tomi.valkeinen@...com>
Subject: Re: CTM and other color related properties

On Wed, Mar 29, 2017 at 10:23:54AM +0300, Jyri Sarha wrote:
> Referring to this discussion:
> https://patchwork.kernel.org/patch/9546905/
> 
> Since the discussion, has there been any planning/work been done about
> the CTM2 API?
> 
> We would need for omap drm (for DSS5 and DSS6) a similar matrix API
> for two purposes. However, neither of them is an exact match to the
> CTM property.
> 
> 1. CRTC specific Color Phase Rotation matrix is pretty close to CTM
> concept, but it is applied after the gamma correction. However, there
> is an optional static full-range or limited-range post-offset vector
> and with it the CTM can also be used to convert the RGB output to a
> YCbCr display output.

Having a post-gamma csc is defintiely the right way to do it. In our
case we don't have that in the current hardware :( All we have is
degamma+csc+gamma, so this complicates things quite a bit when the user
wants to apply ctm and/or gamma and we also need to use the csc either
for rgb->ycbcr or rgb full->limited range conversion. ATM we don't do
ycbcr output (but Shashank has plans) and it looks like our code for
dealing with the rgb full->limited range conversion is totally bogus if
there's a user specified ctm or gamma.

So I think what we want is a degamma->csc->gamma->csc type of
pipeline, where each driver can obviously select which parts of the
pipeline they actually can support.

> 
> 2. Plane specific Color Space Conversion matrix and pre-offset vector
> is for YUV to RGB conversion. For customization purposes we would like
> to expose this 3x3 matrix and the 3-element offset vector to user
> space. So in general this is almost the same thing at the previous, but
> for reverse conversion.

Yeah, for planes I think we want a csc->degamma->csc->gamma type of
pipeline. Again not all hardware can do it all so some of it will be
optional. And on a lot of hardware some of these are totally fixed
function blocks, so eg. exposing a fully programmable matrix may not
always make sense.

We did discuss this on the list recently:
https://lists.freedesktop.org/archives/dri-devel/2017-January/131175.html
https://lists.freedesktop.org/archives/dri-devel/2017-March/135854.html

> 
> So when adding a CTM2 property blob, I would also vote for adding
> pre- and post-offset vectors.

Indeed. I was actually thinking that wouldn't it be cool if the hw
actually had a 4x4 matrix so that you could do the translations purely
with the matrix itself. But I've never actually seen that in any
hardware, so exposing the pre/post offsets separately seems like the
better plan.

> Then a CSC would simply be a
> combination off CTM and either a pre- or post- offset vector or maybe
> both, depending on whether the block provides a conversion from RGB to
> YUV, the other way around, or both.
> 
> Then it is a question whether the offset vectors should be absolute or
> or relative to the bit depth of RGB components. A relative, with enough
> precision, would be the most generic choice but it would leave a lot of
> work to the driver code in many cases.

The actual depth of the data going through the matrix is hardware
dependant anyway, so I don't think absolute values will really work.

> 
> For convenience there could also be a standard enum for selecting
> either custom coefficients or predefined standard conversion
> (Full-range, ITU-R BT.601, and ITU-R BT.709 at least).
> 
> In general the color space conversion arithmetic are described well
> on this web page:
> http://www.equasys.de/colorconversion.html
> 
> Best regards,
> Jyri

-- 
Ville Syrjälä
Intel OTC