linux-kernel - Re: [RFC 0/5] Split iMX8MQ Hantro VPU into G1 and G2 with blk-ctrl support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAHCN7xKb4YhH83oOnbXtkAKWk-3QWNNxE8EhVJ=i7xKW0wjYXw@mail.gmail.com>
Date:   Mon, 6 Dec 2021 16:30:25 -0600
From:   Adam Ford <aford173@...il.com>
To:     Nicolas Dufresne <nicolas@...fresne.ca>
Cc:     linux-media <linux-media@...r.kernel.org>,
        Benjamin Gaignard <benjamin.gaignard@...labora.com>,
        Chris Healy <cphealy@...il.com>,
        Rob Herring <robh+dt@...nel.org>,
        Shawn Guo <shawnguo@...nel.org>,
        Sascha Hauer <s.hauer@...gutronix.de>,
        Pengutronix Kernel Team <kernel@...gutronix.de>,
        Fabio Estevam <festevam@...il.com>,
        NXP Linux Team <linux-imx@....com>,
        Ezequiel Garcia <ezequiel@...guardiasur.com.ar>,
        Philipp Zabel <p.zabel@...gutronix.de>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Lucas Stach <l.stach@...gutronix.de>,
        devicetree <devicetree@...r.kernel.org>,
        arm-soc <linux-arm-kernel@...ts.infradead.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        "open list:HANTRO VPU CODEC DRIVER" 
        <linux-rockchip@...ts.infradead.org>,
        "open list:STAGING SUBSYSTEM" <linux-staging@...ts.linux.dev>
Subject: Re: [RFC 0/5] Split iMX8MQ Hantro VPU into G1 and G2 with blk-ctrl support

On Mon, Dec 6, 2021 at 2:51 PM Nicolas Dufresne <nicolas@...fresne.ca> wrote:
>
> Le dimanche 05 décembre 2021 à 13:51 -0600, Adam Ford a écrit :
> > On Sun, Dec 5, 2021 at 12:16 PM Adam Ford <aford173@...il.com> wrote:
> > >
> > > Currently, the VPU in the i.MQ8MQ is appearing as one codec, but in reality, it's
> > > two IP blocks called G1 and G2.  There is initialization code in VPU code to
> > > pull some clocks, resets and other features which has been integrated into
> > > the vpu-blk-ctrl for the i.MX8M Mini and a similar method can be used to make
> > > the VPU codec's operate as stand-alone cores without having to know the details
> > > of each other or the quirks unique to the i.MX8MQ, so the remaining code can be
> > > left more generic.
> > >
> > > This series was started by Lucas Stach, and picked up by me so some patches have
> > > his s-o-b and mine where I might have changed a few minor items.  It's in an RFC state
> > > because the G2 VP9 operations appear to hang, but the parent code from which I started doesn't
> > > appear to show VP9 support, and it looks like it should.
> > >
> > > since the g-streamer and media trees are in a constant state of change, this series is based on
> > > git://linuxtv.org/hverkuil/media_tree.git for-v5.17e
> > >
> >
> > I forgot to post Fluster results.
> >
> > Before the patches to this branch:
> >
> > 2gst-main] root@...alhost:~/gstreamer/fluster# ./fluster.py list -c |grep -i v4l
> >
> >     GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> >     GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> >     GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> >     GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > GStreamer 1.0... ❌
> > [gst-main] root@...alhost:~/gstreamer/fluster#
> >
> > There was no VP9 support.
> >
> > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> > GStreamer-H.264-V4L2SL-Gst1.0
> > Ran 90/135 tests successfully               in 58.787 secs
> >
> > AFTER this series, two decoders appear:
> >
> > [   15.919137] hantro-vpu 38300000.video-codec: registered
> > nxp,imx8mq-vpu-g1-dec as /dev/video0
> > [   15.983579] hantro-vpu 38310000.video-codec: registered
> > nxp,imx8mq-vpu-g2-dec as /dev/video1
> >
> > VP9 was listed:
> >
> >     GStreamer-H.264-V4L2-Gst1.0: GStreamer H.264 V4L2 decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-H.264-V4L2SL-Gst1.0: GStreamer H.264 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> >     GStreamer-H.265-V4L2-Gst1.0: GStreamer H.265 V4L2 decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for
> > GStreamer 1.0... ❌
> >     GStreamer-VP8-V4L2-Gst1.0: GStreamer VP8 V4L2 decoder for GStreamer 1.0... ❌
> >     GStreamer-VP8-V4L2SL-Gst1.0: GStreamer VP8 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> >     GStreamer-VP9-V4L2SL-Gst1.0: GStreamer VP9 V4L2SL decoder for
> > GStreamer 1.0... ✔️
> >     GStreamer-AV1-V4L2SL-Gst1.0: GStreamer AV1 V4L2SL decoder for
> > GStreamer 1.0... ❌
> >
> > ./fluster.py run -d GStreamer-VP8-V4L2SL-Gst1.0
> > Ran 55/61 tests successfully               in 8.565 secs
> >
> >
> > ./fluster.py run -dGStreamer-H.264-V4L2SL-Gst1.0
> > Ran 90/135 tests successfully               in 60.269 secs
> >
> > Same results for H.264
> >
> > VP9 Hangs, where it didn't even appear as available before:
> >
> > ./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0
> > ****************************************************************************************************
> > Running test suite VP9-TEST-VECTORS with decoder GStreamer-VP9-V4L2SL-Gst1.0
> > Using 4 parallel job(s)
> > ****************************************************************************************************
> >
> > [TEST SUITE      ] (DECODER                    ) TEST VECTOR
> >                           ... RESULT
> > ----------------------------------------------------------------------
> > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > vp90-2-00-quantizer-00.webm                     ... Success
> > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > vp90-2-00-quantizer-01.webm                     ... Success
> > [VP9-TEST-VECTORS] (GStreamer-VP9-V4L2SL-Gst1.0)
> > vp90-2-00-quantizer-02.webm                     ... Succes
> >
> > <hang > - and yes, 'Success' didnt' finish writing to the serial port.
>
> Looks like hope to me ! Do you get further with -j 1 (one concurrent decode) ?

Anything after the first attempt appears to fail.  I haven't figured
out if it's the G2 code or the power domain stuff.  Since VP9 didn't
work I migrated it to use the vpu-blk-ctrl, I can't really compare to
know if it was a regression or an improvement.

:-)

adam

>
> >
> >
> >
> > > Adam Ford (2):
> > >   media: hantro: split i.MX8MQ G1 and G2 code
> > >   arm64: dts: imx8mq: Split i.MX8MQ G1 and G2 with vpu-blk-ctrl
> > >
> > > Lucas Stach (3):
> > >   dt-bindings: power: imx8mq: add defines for VPU blk-ctrl domains
> > >   dt-bindings: soc: add binding for i.MX8MQ VPU blk-ctrl
> > >   soc: imx: imx8m-blk-ctrl: add i.MX8MQ VPU blk-ctrl
> > >
> > >  .../soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml      |  71 +++++++++++
> > >  arch/arm64/boot/dts/freescale/imx8mq.dtsi     |  69 ++++++----
> > >  drivers/soc/imx/imx8m-blk-ctrl.c              |  67 ++++++++++
> > >  drivers/staging/media/hantro/hantro_drv.c     |   4 +-
> > >  drivers/staging/media/hantro/hantro_hw.h      |   2 +-
> > >  drivers/staging/media/hantro/imx8m_vpu_hw.c   | 119 +++---------------
> > >  include/dt-bindings/power/imx8mq-power.h      |   3 +
> > >  7 files changed, 205 insertions(+), 130 deletions(-)
> > >  create mode 100644 Documentation/devicetree/bindings/soc/imx/fsl,imx8mq-vpu-blk-ctrl.yaml
> > >
> > > --
> > > 2.32.0
> > >
>