Message-ID: <20230721181951.GL12001@pengutronix.de>
Date:   Fri, 21 Jul 2023 20:19:51 +0200
From:   Michael Grzeschik <mgr@...gutronix.de>
To:     Paul Kocialkowski <paul.kocialkowski@...tlin.com>
Cc:     linux-kernel@...r.kernel.org, linux-media@...r.kernel.org,
        Hans Verkuil <hverkuil@...all.nl>,
        Sakari Ailus <sakari.ailus@....fi>,
        Nicolas Dufresne <nicolas.dufresne@...labora.com>,
        Andrzej Pietrasiewicz <andrzej.p@...labora.com>,
        Michael Tretter <m.tretter@...gutronix.de>,
        Jernej Škrabec <jernej.skrabec@...il.com>,
        Chen-Yu Tsai <wens@...e.org>,
        Samuel Holland <samuel@...lland.org>,
        Thomas Petazzoni <thomas.petazzoni@...tlin.com>
Subject: Re: Stateless Encoding uAPI Discussion and Proposal

Hi everyone!

Just to let you know. I have just pushed a Branch that includes some first
steps to make the h264-stateless encoder working in Gstreamer. The work is
based on the VP8 Stateless Encoder patches Benjamin Gaignard created.

https://gitlab.freedesktop.org/mgrzeschik/gstreamer/-/commits/1.22/topic/h264-stateless-encoder

The codec this is used with, is the rkvenc that can be found on rockchip
rk3568. I will send an RFC driver for that in the next weeks after my vacation.

On Tue, Jul 11, 2023 at 07:12:41PM +0200, Paul Kocialkowski wrote:
>After various discussions following Andrzej's talk at EOSS, feedback from the
>Media Summit (which I could not attend unfortunately) and various direct
>discussions, I have compiled some thoughts and ideas about stateless encoders
>support with various proposals. This is the result of a few years of interest
>in the topic, after working on a PoC for the Hantro H1 using the hantro driver,
>which turned out to have numerous design issues.
>
>I am now working on a H.264 encoder driver for Allwinner platforms (currently
>focusing on the V3/V3s), which already provides some usable bitstream and will
>be published soon.
>
>This is a very long email where I've tried to split things into distinct topics
>and explain a few concepts to make sure everyone is on the same page.
>
># Bitstream Headers
>
>Stateless encoders typically do not generate all the bitstream headers and
>sometimes no header at all (e.g. Allwinner encoder does not even produce slice
>headers). There's often some hardware block that makes bit-level writing to the
>destination buffer easier (deals with alignment, etc).
>
>The values of the bitstream headers must be in line with how the compressed
>data bitstream is generated and generally follow the codec specification.
>Some encoders might allow configuring all the fields found in the headers,
>others may only allow configuring a few or have specific constraints regarding
>which values are allowed.
>
>As a result, we cannot expect that any given encoder is able to produce frames
>for any set of headers. Reporting related constraints and limitations (beyond
>profile/level) seems quite difficult and error-prone.
>
>So it seems that keeping header generation in-kernel only (close to where the
>hardware is actually configured) is the safest approach.

For the case with the rkvenc, the headers are also not created by the
kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions
that are part of the codecparsers module.

># Codec Features
>
>Codecs have many variable features that can be enabled or not and specific
>configuration fields that can take various values. There is usually some
>top-level indication of profile/level that restricts what can be used.
>
>This is a very similar situation to stateful encoding, where codec-specific
>controls are used to report and set profile/level and configure these aspects.
>A particularly nice thing about it is that we can reuse these existing controls
>and add new ones in the future for features that are not yet covered.
>
>This approach feels more flexible than designing new structures with a selected
>set of parameters (that could match the existing controls) for each codec.

I back the Idea of generic profiles instead of explicit configuration
from the usersapace point of view.

The parameterization works like this:

Read the sane default parameter set from the driver.
Modify the parameters based on the userspace decisions.
 - (currently hardcoded and not based on any user input)
Write the updated parameters back to the driver.

># Reference and Reconstruction Management
<snip>

># Frame Types
<snip>

># Rate Control
<snip>

># Regions of Interest
<snip>

Since the first shot of the rkvenc is I-Frame only code, these other topics are
currently undefined and unimplemented in the Gstreamer stack.


Regards,
Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)

Powered by blists - more mailing lists