Message-ID: <20250422140055.64b98cc1@eldfell>
Date: Tue, 22 Apr 2025 14:00:55 +0300
From: Pekka Paalanen <ppaalanen@...il.com>
To: Geert Uytterhoeven <geert@...ux-m68k.org>
Cc: Laurent Pinchart <laurent.pinchart@...asonboard.com>, Tomi Valkeinen
 <tomi.valkeinen@...asonboard.com>, Vishal Sagar <vishal.sagar@....com>,
 Anatoliy Klymenko <anatoliy.klymenko@....com>, Maarten Lankhorst
 <maarten.lankhorst@...ux.intel.com>, Maxime Ripard <mripard@...nel.org>,
 Thomas Zimmermann <tzimmermann@...e.de>, David Airlie <airlied@...il.com>,
 Simona Vetter <simona@...ll.ch>, Michal Simek <michal.simek@....com>,
 dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org,
 linux-arm-kernel@...ts.infradead.org, Dmitry Baryshkov
 <dmitry.baryshkov@....qualcomm.com>
Subject: Re: [PATCH v4 03/11] drm/fourcc: Add DRM_FORMAT_Y8

On Tue, 22 Apr 2025 12:12:21 +0200
Geert Uytterhoeven <geert@...ux-m68k.org> wrote:

> Hi Pekka,
> 
> On Tue, 22 Apr 2025 at 12:01, Pekka Paalanen <ppaalanen@...il.com> wrote:
> > On Tue, 22 Apr 2025 11:41:29 +0200
> > Geert Uytterhoeven <geert@...ux-m68k.org> wrote:  
> > > On Tue, 22 Apr 2025 at 11:11, Pekka Paalanen
> > > <pekka.paalanen@...oniitty.fi> wrote:  
> > > > On Mon, 21 Apr 2025 17:50:39 +0300
> > > > Laurent Pinchart <laurent.pinchart@...asonboard.com> wrote:  
> > > > > On Thu, Apr 17, 2025 at 11:13:15AM +0300, Pekka Paalanen wrote:  
> > > > > > On Wed, 16 Apr 2025 11:59:43 +0300 Tomi Valkeinen wrote:  
> > > > > > > On 01/04/2025 16:27, Pekka Paalanen wrote:  
> > > > > > > > On Mon, 31 Mar 2025 13:53:37 +0300 Pekka Paalanen wrote:  
> > > > > > > >> On Mon, 31 Mar 2025 11:21:35 +0300 Laurent Pinchart wrote:  
> > > > > > > >>> On Mon, Mar 31, 2025 at 10:54:46AM +0300, Pekka Paalanen wrote:  
> > > > > > > >>>> On Thu, 27 Mar 2025 17:35:39 +0100 Geert Uytterhoeven wrote:  
> > > > > > > >>>>> On Thu, 27 Mar 2025 at 16:59, Pekka Paalanen wrote:  
> > > > > > > >>>>>> On Thu, 27 Mar 2025 16:21:16 +0200 Tomi Valkeinen wrote:  
> > > > > > > >>>>>>> On 27/03/2025 11:20, Pekka Paalanen wrote:  
> > > > > > > >>>>>>>> On Wed, 26 Mar 2025 15:55:18 +0200 Tomi Valkeinen wrote:  
> > > > > > > >>>>>>>>> On 26/03/2025 15:52, Geert Uytterhoeven wrote:  
> > > > > > > >>>>>>>>>> On Wed, 26 Mar 2025 at 14:23, Tomi Valkeinen wrote:  
> > > > > > > >>>>>>>>>>> Add greyscale Y8 format.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Acked-by: Dmitry Baryshkov <dmitry.baryshkov@...aro.org>
> > > > > > > >>>>>>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@...asonboard.com>  
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Thanks for your patch!
> > > > > > > >>>>>>>>>>  
> > > > > > > >>>>>>>>>>> --- a/include/uapi/drm/drm_fourcc.h
> > > > > > > >>>>>>>>>>> +++ b/include/uapi/drm/drm_fourcc.h
> > > > > > > >>>>>>>>>>> @@ -405,6 +405,9 @@ extern "C" {
> > > > > > > >>>>>>>>>>>     #define DRM_FORMAT_YUV444      fourcc_code('Y', 'U', '2', '4') /* non-subsampled Cb (1) and Cr (2) planes */
> > > > > > > >>>>>>>>>>>     #define DRM_FORMAT_YVU444      fourcc_code('Y', 'V', '2', '4') /* non-subsampled Cr (1) and Cb (2) planes */
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> +/* Greyscale formats */
> > > > > > > >>>>>>>>>>> +
> > > > > > > >>>>>>>>>>> +#define DRM_FORMAT_Y8          fourcc_code('G', 'R', 'E', 'Y')  /* 8-bit Y-only */  
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> This format differs from e.g. DRM_FORMAT_R8, which encodes
> > > > > > > >>>>>>>>>> the number of bits in the FOURCC format. What do you envision
> > > > > > > >>>>>>>>>> for e.g. DRM_FORMAT_Y16? fourcc_code('G', 'R', '1', '6')?  
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> I wanted to use the same fourcc as on V4L2 side. Strictly speaking it's
> > > > > > > >>>>>>>>> not required, but different fourccs for the same formats do confuse.
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> So, generally speaking, I'd pick an existing fourcc from v4l2 side if
> > > > > > > >>>>>>>>> possible, and if not, invent a new one.  
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> what's the actual difference between DRM_FORMAT_R8 and DRM_FORMAT_Y8?
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Is the difference that when R8 gets expanded to RGB, it becomes (R, 0,
> > > > > > > >>>>>>>> 0), but Y8 gets expanded to (c1 * Y, c2 * Y, c3 * Y) where c1..c3 are
> > > > > > > >>>>>>>> defined by MatrixCoefficients (H.273 terminology)?
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> That would be my intuitive assumption following how YCbCr is handled.
> > > > > > > >>>>>>>> Is it obvious enough, or should there be a comment to that effect?  
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> You raise an interesting point. Is it defined how a display driver, that
> > > > > > > >>>>>>> supports R8 as a format, shows R8 on screen? I came into this in the
> > > > > > > >>>>>>> context of grayscale formats, so I thought R8 would be handled as (R, R,
> > > > > > > >>>>>>> R) in RGB. But you say (R, 0, 0), which... also makes sense.  
> > > > > > > >>>>>>
> > > > > > > >>>>>> That is a good question too. I based my assumption on OpenGL behavior
> > > > > > > >>>>>> of R8.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Single channel displays do exist I believe, but being single-channel,
> > > > > > > >>>>>> expansion on the other channels is likely meaningless. Hm, but for the
> > > > > > > >>>>>> KMS color pipeline, it would be meaningful, like with a CTM.
> > > > > > > >>>>>> Interesting.
> > > > > > > >>>>>>
> > > > > > > >>>>>> I don't know. Maybe Geert does?  
> > > > > > > >>>>>
> > > > > > > >>>>> I did some digging, and was a bit surprised that it was you who told
> > > > > > > >>>>> me to use R8 instead of Y8?
> > > > > > > >>>>> https://lore.kernel.org/all/20220202111954.6ee9a10c@eldfell  
> > > > > > > >>>>
> > > > > > > >>>> Hi Geert,
> > > > > > > >>>>
> > > > > > > >>>> indeed I did. I never thought of the question of expansion to R,G,B
> > > > > > > >>>> before. Maybe that expansion is what spells R8 and Y8 apart?
> > > > > > > >>>>
> > > > > > > >>>> I do think that expansion needs to be specified, so that the KMS color
> > > > > > > >>>> pipeline computations are defined. There is a big difference between
> > > > > > > >>>> multiplying these with an arbitrary 3x3 matrix (e.g. CTM):
> > > > > > > >>>>
> > > > > > > >>>> - (R, 0, 0)
> > > > > > > >>>> - (R, R, R)
> > > > > > > >>>> - (c1 * Y, c2 * Y, c3 * Y)  
> > > > > > > >>>
> > > > > > > >>> I'd be very surprised by an YUV to RGB conversion matrix where the first
> > > > > > > >>> column would contain different values. What we need to take into account
> > > > > > > >>> though is quantization (full vs. limited range).  
> > > > > > > >
> > > > > > > > Quantization range is indeed good to note. R8 would be always full
> > > > > > > > range, but Y8 would follow COLOR_RANGE property.
> > > > > > > >  
> > > > > > > >> That makes Y8 produce (Y, Y, Y), and we have our answer: R8 should be
> > > > > > > >> (R, 0, 0), so we have both variants.
> > > > > > > >>
> > > > > > > >> Can we specify Y, R, G and B be nominal values in the range 0.0 - 1.0
> > > > > > > >> in the KMS color processing?  
> > > > > > > >
> > > > > > > > I think this 0.0 - 1.0 nominal range definition for the abstract KMS
> > > > > > > > color processing is necessary.
> > > > > > > >
> > > > > > > > It also means that limited range Y8 data, when containing values 0-15
> > > > > > > > or 240-255, would produce negative and greater than 1.0 values,
> > > > > > > > respectively. They might get immediately clamped to 0.0 - 1.0 with the
> > > > > > > > first color operation they face, though, but the concept seems
> > > > > > > > important and carrying over to the new color pipelines UAPI which might
> > > > > > > > choose not to clamp.  
> > > > > > >
> > > > > > > Is the behavior of values outside the limited range something that needs
> > > > > > > to be defined? We can't know how each piece of HW behaves with
> > > > > > > "undefined" input, so should we not just define the behavior as platform
> > > > > > > specific?  
> > > > > >
> > > > > > Hi Tomi,
> > > > > >
> > > > > > it's not undefined nor illegal input in general. The so-called
> > > > > > sub-black and super-white ranges exist for a reason, and they are
> > > > > > intended to be used in video processing to avoid clipping in
> > > > > > intermediate processing steps when a filter overshoots a bit. There are
> > > > > > also practices that depend on them, like PLUGE calibration with
> > > > > > traditional signals on a display: https://www.itu.int/rec/R-REC-BT.814
> > > > > >
> > > > > > I think it would be really good to have defined behaviour if at all
> > > > > > possible.
> > > > > >  
> > > > > > > In any case: I can't say I fully understood all the discussions wrt.
> > > > > > > color spaces. But my immediate interest is, of course, this series =).
> > > > > > > So is there something that you think should be improved here?  
> > > > > >
> > > > > > Right, the range discussion is a tangent and applies to all YUV
> > > > > > formats, so it's not a new question.
> > > > > >  
> > > > > > > My understanding is that the Y-only pixel formats behave in a well
> > > > > > > defined way (or, as well defined as the YUV formats), and there's
> > > > > > > nothing more to add here. Is that right?  
> > > > > >
> > > > > > There are two things:
> > > > > >
> > > > > > - Y8 follows COLOR_RANGE property, just like all other YUV formats.
> > > > > > - Y8 implies that Cb and Cr are both neutral (0.0 in nominal values).
> > > > > >
> > > > > > I'd like these explicitly written down, so that they become obvious to
> > > > > > everyone. I suspect either one might be easy to forget when writing
> > > > > > code and taking shortcuts without thinking.
> > > > > >
> > > > > >
> > > > > > Laurent,
> > > > > >
> > > > > > I did find a case where (Y', neutral, neutral) does *not* seem to expand
> > > > > > to RGB=(Y, Y, Y): ICtCp. The conversion from ICtCp to L'M'S' does
> > > > > > produce (Y', Y', Y'), but the LMS-to-RGB matrix scrambles it.
> > > > > >
> > > > > > I didn't dig through BT.2020 constant-luminance Y'C'bcC'rc, but I
> > > > > > wouldn't be surprised if it scrambled too.
> > > > > >
> > > > > > Of course, both of the above are not just one matrix. They require two
> > > > > > matrices and the transfer characteristic each to compute. KMS color
> > > > > > operations cannot implement those today, but with the colorop pipelines
> > > > > > they will if the hardware does it.
> > > > > >
> > > > > > That's why I think it's important to document the assumption of Cb and
> > > > > > Cr when not part of the pixel format, and not write down a specific
> > > > > > expansion to RGB like (Y, Y, Y).  
> > > > >
> > > > > Every time I discuss color spaces, the scopes of "RGB" and "YUV" seem to
> > > > > expand more and more. This makes me wonder how we define those two
> > > > > concepts. Taking the conversion from RGB to ICtCp as an example, would
> > > > > you consider LMS and L'M'S' as "RGB" formats, and ICtCp as a "YUV"
> > > > > format ?  
> > > >
> > > > sorry for the confusion. In this specific context, my use of RGB and
> > > > YUV refers to the channels in DRM pixel formats. It might have been
> > > > better if all channels in all pixel formats were "anonymous" and merely
> > > > numbered because all formats can be used for any color model, but this
> > > > is what we have.
> > > >
> > > > There is some disambiguation in
> > > > https://gitlab.freedesktop.org/pq/color-and-hdr/-/blob/main/doc/pixels_color.md
> > > > The doc is some years old, so nowadays I might phrase things
> > > > differently, but maybe it's easier to read for those new to things as I
> > > > wrote it when I was just learning things.
> > > >
> > > > I would classify ICtCp in the YUV pixel format category, because the
> > > > CtCp plane can be sub-sampled (right?). I would classify LMS and L'M'S'
> > > > in the RGB pixel format category because they are not sub-sampled AFAIK
> > > > although they also do not actually appear as buffer contents, so the
> > > > relation to pixel formats is... theoretical.
> > > >
> > > > IOW, we have a completely artificial split of DRM pixel formats to RGB
> > > > and YUV where the only essential difference is that YUV formats can have
> > > > sub-sampled variants and RGB formats do not.  
> > >
> > > RGB can be subsampled, too...
> > > https://en.wikipedia.org/wiki/Bayer_filter  
> >
> > That's true. What difference are we left with, then?  
> 
> RGB contains three monochromatic color channels (which may differ from
> Red, Green, and Blue, cfr. truncated RGn and Rn formats).
> YUV contains one luminance and two chrominance channels.

I think I get what you mean. RGB are directly related to the power of
each of the primary emitters in a display. YUV OTOH carries the overall
luma and the difference from neutral (chroma) signals. This difference
is the difference between color models.

Monochromatic is not the right word here, unless you explicitly refer
to the primaries of BT.2020 which are laser-like. Most display
primaries are far from monochromatic. The closer to monochromatic
display primaries get, the more pronounced the differences between
color vision in people become, even if the people were all classified
as having normal color vision.


Thanks,
pq

> > We have DRM pixel formats which imply some color model, but do not
> > define the variant of each color model (e.g. the Y'CbCr-to-RGB matrix).
> >
> > I guess the implied color model then implies which API, e.g. KMS plane
> > property, is responsible for defining the variant.  
> 
> Probably (IANADX --- I Am Not A Drm eXpert ;-)
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 


Content of type "application/pgp-signature" skipped

Powered by blists - more mailing lists