linux-kernel - Re: [PATCH v2 7/8] media: hevc: Add scaling matrix control

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 14 Jun 2021 09:50:46 -0300
From:   Ezequiel Garcia <ezequiel@...labora.com>
To:     Benjamin Gaignard <benjamin.gaignard@...labora.com>,
        Hans Verkuil <hverkuil@...all.nl>, p.zabel@...gutronix.de,
        mchehab@...nel.org, shawnguo@...nel.org, s.hauer@...gutronix.de,
        festevam@...il.com, gregkh@...uxfoundation.org, mripard@...nel.org,
        paul.kocialkowski@...tlin.com, wens@...e.org,
        jernej.skrabec@...l.net, emil.l.velikov@...il.com,
        andrzej.p@...labora.com, jc@...esim.co.uk, jernej.skrabec@...il.com
Cc:     kernel@...gutronix.de, linux-imx@....com,
        linux-media@...r.kernel.org, linux-rockchip@...ts.infradead.org,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2 7/8] media: hevc: Add scaling matrix control

On Mon, 2021-06-14 at 09:43 +0200, Benjamin Gaignard wrote:
> 
> Le 14/06/2021 à 09:27, Hans Verkuil a écrit :
> > On 10/06/2021 17:44, Benjamin Gaignard wrote:
> > > HEVC scaling lists are used for the scaling process for transform
> > > coefficients.
> > > V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED has to set when they are
> > > encoded in the bitstream.
> > Comparing H264 with HEVC I noticed that the corresponding flag for H264 is
> > called V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT.
> > 
> > Should those names be aligned? Also, it is part of PPS for H264 and SPS in HEVC,
> > is that difference correct?
> 
> In ITU specifications ("7.4.3.2.1 General sequence parameter set RBSP semantics") this flag is define like that:
> scaling_list_enabled_flag equal to 1 specifies that a scaling list is used for the scaling process for transform coefficients.
> scaling_list_enabled_flag equal to 0 specifies that scaling list is not used for the scaling process for transform coefficients.
> 
> So for me the naming is correct.
> 

The bitstream is really parsed in userspace (gstreamer, ffmpeg, chromium).
Not all bitstream parameters need to be passed as-is, because the kernel
is actually a sort of low-level layer in the decoder stack.

I think it's probably most appropriate to follow our H.264 interface
semantics, see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=54889c51b833d236228f983be16212fbe806bb89.

The H.264 story goes like this:

* Default scaling list is used for decoding, but can be modified
  by a bitstream-provided scaling list.

* The scaling list modification can be in SPS or in PPS.

* For each frame, the SPS and PPS headers will tell if
  a modified scaling list must be used for decoding,
  and if it's provided in the SPS header or the PPS header.

The userspace parser must take care of this, and pass
a scaling matrix to the kernel via V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX,
setting PPS_FLAG_SCALING_MATRIX_PRESENT.

This flag is at the PPS control, so the scaling modification
can be applied on each frame.

In other words, the kernel interface is simpler, it just
receives a scaling matrix and a flag telling if it must be used or not. 

We should probably clarify the documentation, so this process is more clear.

>From the HEVC spec, it seems the process should be similar.
The only difference is the SPS "scaling_list_enabled_flag" parameter,
which doesn't exist in H.264.

Nicolas: what do you think?

Thanks,
Ezequiel