Message-ID: <eef89bea-244f-fa20-08f0-50cb1e03bede@collabora.com>
Date:   Mon, 7 Mar 2022 09:49:16 +0100
From:   Benjamin Gaignard <benjamin.gaignard@...labora.com>
To:     Adam Ford <aford173@...il.com>
Cc:     Mauro Carvalho Chehab <mchehab@...nel.org>,
        Ezequiel Garcia <ezequiel@...guardiasur.com.ar>,
        Philipp Zabel <p.zabel@...gutronix.de>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        mripard@...nel.org, paul.kocialkowski@...tlin.com,
        Chen-Yu Tsai <wens@...e.org>,
        "jernej.skrabec" <jernej.skrabec@...il.com>,
        Jonas Karlman <jonas@...boo.se>,
        Nicolas Dufresne <nicolas@...fresne.ca>,
        linux-media <linux-media@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        "open list:STAGING SUBSYSTEM" <linux-staging@...ts.linux.dev>,
        arm-soc <linux-arm-kernel@...ts.infradead.org>,
        linux-sunxi@...ts.linux.dev, kernel <kernel@...labora.com>,
        knaerzche@...il.com, jc@...esim.co.uk,
        Chris Healy <cphealy@...il.com>,
        Ruslan Sushko <ruslan.sushko@....aero>
Subject: Re: [PATCH v3 00/14] Move HEVC stateless controls out of staging


Le 04/03/2022 à 23:46, Adam Ford a écrit :
> On Fri, Mar 4, 2022 at 7:03 AM Benjamin Gaignard
> <benjamin.gaignard@...labora.com> wrote:
>>
>> Le 04/03/2022 à 13:59, Adam Ford a écrit :
>>> On Thu, Mar 3, 2022 at 4:13 AM Benjamin Gaignard
>>> <benjamin.gaignard@...labora.com> wrote:
>>>> Le 03/03/2022 à 02:23, Adam Ford a écrit :
>>>>> On Mon, Feb 28, 2022 at 4:13 AM Benjamin Gaignard
>>>>> <benjamin.gaignard@...labora.com> wrote:
>>>>>> Le 26/02/2022 à 23:25, Adam Ford a écrit :
>>>>>>> On Fri, Feb 25, 2022 at 4:41 PM Benjamin Gaignard
>>>>>>> <benjamin.gaignard@...labora.com> wrote:
>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware
>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI).
>>>>>>>>
>>>>>>>> After the remarks done on version 2, I have completely reworked to patches
>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the
>>>>>>>> titles.
>>>>>>>>
>>>>>>>> In this v3 I do all the changes (new controls, documentation, etc..)
>>>>>>>> in the staging directory before moving the HEVC uAPI to stable
>>>>>>>> steps by steps (unlike the big one patch in v2).
>>>>>>>>
>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>>>>>> I have push a branch here:
>>>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
>>>>>>
>>>>>> it is the incoming version 4 of this series + patches to enable G2 on my IMX8MQ
>>>>> Benjamin,
>>>>>
>>>>> I checked this repo out, and built it along with pulling the latest
>>>>> versions of G-streamer and fluster.
>>>>> When I check for v4l2 compatibility, I get the following:
>>>>>
>>>>>        GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
>>>>> GStreamer 1.0... ❌
>>>>>        GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
>>>>> GStreamer 1.0... ✔️
>>>>>        GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
>>>>> GStreamer 1.0... ❌
>>>>>        GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
>>>>> GStreamer 1.0... ❌
>>>>>        GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
>>>>> GStreamer 1.0... ❌
>>>>>        GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
>>>>>        GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
>>>>> GStreamer 1.0... ✔️
>>>>>        GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
>>>>> GStreamer 1.0... ✔️
>>>>>
>>>>> I see H,264, VP8 and VP9, but I would have also expected
>>>>> GStreamer-H.265-V4L2SL-Gst1.0 to return with a check box.
>>>>>
>>>>> When I checked to see if both decoders were being enumerated, I found they were.
>>>>> [gst-main] root@...alhost:~/gstreamer/fluster# dmesg |grep -i hantro
>>>>> [   16.044243] hantro_vpu: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [   16.044243] hantro_vpu: module is from the staging directory, the
>>>>> quality is unknown, you have been warned.
>>>>> [   16.095661] hantro-vpu 38300000.video-codec: registered
>>>>> nxp,imx8mq-vpu-g1-dec as /dev/video0
>>>>> [   16.096782] hantro-vpu 38310000.video-codec: registered
>>>>> nxp,imx8mq-vpu-g2-dec as /dev/video1
>>>>>
>>>>> Did I do something wrong, or did I miss something?
>>>> Hi Adam,
>>>>
>>>> I guess it could be a misalignment between v4l2 kernel headers
>>>> and GStreamer v4l2 headers.
>>>> I have push a new version of the GST merge request which is aligned with
>>>> HEVC uAPI v4 proposal.
>>>>
>>>> When you inspect v4l2codecs gst plugin (gst-inspect-1.0 v4l2codecs) do
>>>> you see v4l2slh265dec plugin ?
>>>> I have fluster happy with it:
>>>> ./fluster.py list -c
>>> I pulled the latest from g-streamer's git repo again, and re-built.
>>> gst-instpect does not appear to show the h265 codec anymore.
>>>
>>> # gst-inspect-1.0 |grep v4l2codecs
>>> v4l2codecs:  v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
>>> v4l2codecs:  v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
>>> v4l2codecs:  v4l2slvp8alphadecodebin: VP8 Alpha Decoder
>>> v4l2codecs:  v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
>>> v4l2codecs:  v4l2slvp9alphadecodebin: VP9 Alpha Decoder
>>> v4l2codecs:  v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder
>>>
>>> Is there a specific branch of g-streamer I should be using that can
>>> handle the newer API?
>> You can use this branch:
>>
>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/tree/HEVC_aligned_with_kernel_5.15
>>
>> It is the one used to create the merge request for h265 plugin.
> I used the above gstreamer branch with
> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/HEVC_UAPI_V4
> for the kernel.
>
> I was able to confirm the gst-inspect showed the H265:
>
> [gst-HEAD] root@...alhost:~/fluster# gst-inspect-1.0 |grep 265
> libav:  avdec_h265: libav HEVC (High Efficiency Video Coding) decoder
> rtp:  rtph265depay: RTP H265 depayloader
>
> rtp:  rtph265pay: RTP H265 payloader
> typefindfunctions: video/x-h265: h265, x265, 265
> v4l2codecs:  v4l2slh265dec: V4L2 Stateless H.265 Video Decoder
> videoparsersbad:  h265parse: H.265 parser
>
> Unfortunately, fluster doesn't appear to recognize it:
>
>      GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> GStreamer 1.0... ❌
>
> Attempting to run fluster with that decoder fail:
>
> # ./fluster.py run -d GStreamer-H.265-V4L2SL-Gst1.0 -j1
> Skipping decoder GStreamer-H.265-V4L2SL-Gst1.0 because it cannot be run
> [gst-HEAD] root@...alhost:~/fluster#

Can you try with -vvv option in fluster command line ?

Does a manual gst pipeline with v4l2slh265dec works ?

Regards,
Benjamin

>
>
>
>
>
>
>
>> Benjamin
>>
>> adam
>>
>>>> H265
>>>>        ...
>>>>        GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for GStreamer 1.0... ❌
>>>>        GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for GStreamer 1.0... ✔️
>>>>        GStreamer-H.265-VA-Gst1.0: GStreamer H.265 VA decoder for GStreamer 1.0... ❌
>>>>        GStreamer-H.265-VAAPI-Gst1.0: GStreamer H.265 VAAPI decoder for GStreamer 1.0... ❌
>>>>        JCT-VT-H.265: JCT-VT H.265/HEVC reference decoder... ❌
>>>>
>>>> I hope that will help you,
>>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>> adam
>>>>>
>>>>>> Regards,
>>>>>> Benjamin
>>>>>>
>>>>>>> Benjamin,
>>>>>>>
>>>>>>> I have an imx8mm and imx8mq that I can test. Do you happen to have a
>>>>>>> repo that I can clone to test this?  The imx8m stuff is spread around
>>>>>>> between the media tree and the imx tree since it hasn't been fully
>>>>>>> merged yet.
>>>>>>>
>>>>>>> thanks,
>>>>>>>
>>>>>>> adam
>>>>>>>
>>>>>>>> Benjamin
>>>>>>>>
>>>>>>>> Benjamin Gaignard (11):
>>>>>>>>       media: uapi: HEVC: Add missing fields in HEVC controls
>>>>>>>>       media: uapi: HEVC: Rename HEVC stateless controls with STATELESS
>>>>>>>>         prefix
>>>>>>>>       media: uapi: HEVC: Add document uAPI structure
>>>>>>>>       media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS as a
>>>>>>>>         dynamic array
>>>>>>>>       media: uapi: Move parsed HEVC pixel format out of staging
>>>>>>>>       media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>>>>>>>>       media: uapi: Move the HEVC stateless control type out of staging
>>>>>>>>       media: controls: Log HEVC stateless control in .std_log
>>>>>>>>       media: uapi: Create a dedicated header for Hantro control
>>>>>>>>       media: uapi: HEVC: fix padding in v4l2 control structures
>>>>>>>>       media: uapi: move HEVC stateless controls out of staging
>>>>>>>>
>>>>>>>> Hans Verkuil (3):
>>>>>>>>       videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>>>>>>>>       v4l2-ctrls: add support for dynamically allocated arrays.
>>>>>>>>       vivid: add dynamic array test control
>>>>>>>>
>>>>>>>>      .../userspace-api/media/drivers/hantro.rst    |   5 -
>>>>>>>>      .../media/v4l/ext-ctrls-codec-stateless.rst   | 831 ++++++++++++++++++
>>>>>>>>      .../media/v4l/ext-ctrls-codec.rst             | 780 ----------------
>>>>>>>>      .../media/v4l/pixfmt-compressed.rst           |   7 +-
>>>>>>>>      .../media/v4l/vidioc-g-ext-ctrls.rst          |  20 +
>>>>>>>>      .../media/v4l/vidioc-queryctrl.rst            |   8 +
>>>>>>>>      .../media/videodev2.h.rst.exceptions          |   5 +
>>>>>>>>      .../media/test-drivers/vivid/vivid-ctrls.c    |  15 +
>>>>>>>>      drivers/media/v4l2-core/v4l2-ctrls-api.c      | 103 ++-
>>>>>>>>      drivers/media/v4l2-core/v4l2-ctrls-core.c     | 198 ++++-
>>>>>>>>      drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  32 +-
>>>>>>>>      drivers/media/v4l2-core/v4l2-ctrls-priv.h     |   3 +-
>>>>>>>>      drivers/media/v4l2-core/v4l2-ctrls-request.c  |  13 +-
>>>>>>>>      drivers/staging/media/hantro/hantro_drv.c     |  27 +-
>>>>>>>>      drivers/staging/media/hantro/hantro_hevc.c    |   8 +-
>>>>>>>>      drivers/staging/media/sunxi/cedrus/cedrus.c   |  24 +-
>>>>>>>>      .../staging/media/sunxi/cedrus/cedrus_dec.c   |  10 +-
>>>>>>>>      include/media/hevc-ctrls.h                    | 250 ------
>>>>>>>>      include/media/v4l2-ctrls.h                    |  48 +-
>>>>>>>>      include/uapi/linux/hantro-media.h             |  19 +
>>>>>>>>      include/uapi/linux/v4l2-controls.h            | 436 +++++++++
>>>>>>>>      include/uapi/linux/videodev2.h                |  13 +
>>>>>>>>      22 files changed, 1686 insertions(+), 1169 deletions(-)
>>>>>>>>      delete mode 100644 include/media/hevc-ctrls.h
>>>>>>>>      create mode 100644 include/uapi/linux/hantro-media.h
>>>>>>>>
>>>>>>>> --
>>>>>>>> 2.32.0
>>>>>>>>

Powered by blists - more mailing lists