linux-kernel - RE: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Fri, 3 Dec 2021 05:20:07 +0000
From:   Ming Qian <ming.qian@....com>
To:     Nicolas Dufresne <nicolas@...fresne.ca>,
        "mchehab@...nel.org" <mchehab@...nel.org>,
        "shawnguo@...nel.org" <shawnguo@...nel.org>,
        "robh+dt@...nel.org" <robh+dt@...nel.org>,
        "s.hauer@...gutronix.de" <s.hauer@...gutronix.de>
CC:     "hverkuil-cisco@...all.nl" <hverkuil-cisco@...all.nl>,
        "kernel@...gutronix.de" <kernel@...gutronix.de>,
        "festevam@...il.com" <festevam@...il.com>,
        dl-linux-imx <linux-imx@....com>,
        Aisheng Dong <aisheng.dong@....com>,
        "linux-media@...r.kernel.org" <linux-media@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>
Subject: RE: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder driver

> -----Original Message-----
> From: Nicolas Dufresne [mailto:nicolas@...fresne.ca]
> Sent: Friday, December 3, 2021 12:10 PM
> To: Ming Qian <ming.qian@....com>; mchehab@...nel.org;
> shawnguo@...nel.org; robh+dt@...nel.org; s.hauer@...gutronix.de
> Cc: hverkuil-cisco@...all.nl; kernel@...gutronix.de; festevam@...il.com;
> dl-linux-imx <linux-imx@....com>; Aisheng Dong <aisheng.dong@....com>;
> linux-media@...r.kernel.org; linux-kernel@...r.kernel.org;
> devicetree@...r.kernel.org; linux-arm-kernel@...ts.infradead.org
> Subject: Re: [EXT] Re: [PATCH v12 00/13] amphion video decoder/encoder
> driver
> 
> Caution: EXT Email
> 
> Le vendredi 26 novembre 2021 à 06:31 +0000, Ming Qian a écrit :
> > > -----Original Message-----
> > > From: Nicolas Dufresne [mailto:nicolas@...fresne.ca]
> > > Sent: Thursday, November 25, 2021 11:36 PM
> > > To: Ming Qian <ming.qian@....com>; mchehab@...nel.org;
> > > shawnguo@...nel.org; robh+dt@...nel.org; s.hauer@...gutronix.de
> > > Cc: hverkuil-cisco@...all.nl; kernel@...gutronix.de;
> > > festevam@...il.com; dl-linux-imx <linux-imx@....com>; Aisheng Dong
> > > <aisheng.dong@....com>; linux-media@...r.kernel.org;
> > > linux-kernel@...r.kernel.org; devicetree@...r.kernel.org;
> > > linux-arm-kernel@...ts.infradead.org
> > > Subject: Re: [EXT] Re: [PATCH v12 00/13] amphion video
> > > decoder/encoder driver
> > >
> > > Caution: EXT Email
> > >
> > > Le jeudi 25 novembre 2021 à 05:25 +0000, Ming Qian a écrit :
> > > > For test [JCT-VC-HEVC_V1] (GStreamer-H.265-V4L2-Gst1.0)
> > > > VPSSPSPPS_A_MainConcept_1, The vpu report an unsupported message
> > > > to
> > > driver, so driver report pollerr to gstreamer.
> > > > But this stream can be decoded using the amphion vpu when I test
> > > > it using our unit test, I checked the difference, there are many
> > > > vps, sps and pps at the beginning, gstreamer will skip the first
> > > > vpu and two pps, totally skip 56 bytes. It leds to vpu can't
> > > > decode And our unit test won't
> > > skip anthing, so the vpu can decode the stream.
> > >
> > > This specific test triggers a bug in GStreamer HEVC parser, I'm
> > > aware of this one, and it is on my todo to fix (just not as trivial
> > > as it looks like, the VCL nal detection code was implemented wrong
> > > and that ended up leaking into the rest of the design). This
> > > specific test will of course be marked and skipped for CI test that
> > > uses GStreamer.
> > >
> > > regards,
> > > Nicolas
> >
> > Hi Nicolas,
> >
> >     For the h264 decoder test, If I test with our unit test tool, the
> > result is 124/135 tests successfully, But only 75/135 tests
> > successfully when test using gstreamer.
> >     There are 49 tests with different results, and all of the 49 test
> > streams are interlaced stream.
> > The amphion vpu will output the interlaced frame directly, it won't
> > merge the interlaced two parts into one progressive frame.
> >      And the gstreamer tiled unpack function(unpack_NV12_TILED) won't
> > handle the interlaced case, so the output frame is abnormal, and led to test
> fail.
> >      I think it should be a hardware limitation of the amphion vpu.
> > And maybe
> 
> If this is the case, it needs to be signaled as ALTERNATE (if you have 1 v4l2
> buffer per field) or SEQ if they are stored in the same v4l2_buffer but on top of
> each other instead of interleaved.
> 
> Some bit about this might not we all wired properly, I have only tested
> alternate field for capture devices (Xilinx ZinqMP). Please check the signalling
> of interlace field in your driver, I'll check the state of GStreamer in this regard.


The two fields are stored in the same v4l2_buffer, I'll change the filed to V4L2_FIELD_SEQ_TB and test it again.

> 
> > the gstreamer videoconvert can handle it.
> > The interlaced list is as below:
> > [JVT-AVC_V1] cabac_mot_fld0_full
> > [JVT-AVC_V1] cabac_mot_mbaff0_full
> > [JVT-AVC_V1] cabac_mot_picaff0_full
> > [JVT-AVC_V1] CABREF3_Sand_D
> > [JVT-AVC_V1] CAFI1_SVA_C
> > [JVT-AVC_V1] CAMA1_Sony_C
> > [JVT-AVC_V1] CAMA1_TOSHIBA_B
> > [JVT-AVC_V1] CAMA3_Sand_E
> > [JVT-AVC_V1] CAMACI3_Sony_C
> > [JVT-AVC_V1] CAMANL1_TOSHIBA_B
> > [JVT-AVC_V1] CAMANL2_TOSHIBA_B
> > [JVT-AVC_V1] CAMANL3_Sand_E
> > [JVT-AVC_V1] CAMASL3_Sony_B
> > [JVT-AVC_V1] CAMP_MOT_MBAFF_L30
> > [JVT-AVC_V1] CAMP_MOT_MBAFF_L31
> > [JVT-AVC_V1] CANLMA2_Sony_C
> > [JVT-AVC_V1] CANLMA3_Sony_C
> > [JVT-AVC_V1] CAPA1_TOSHIBA_B
> > [JVT-AVC_V1] CAPAMA3_Sand_F
> > [JVT-AVC_V1] cavlc_mot_fld0_full_B
> > [JVT-AVC_V1] cavlc_mot_mbaff0_full_B
> > [JVT-AVC_V1] cavlc_mot_picaff0_full_B
> > [JVT-AVC_V1] CVCANLMA2_Sony_C
> > [JVT-AVC_V1] CVFI1_Sony_D
> > [JVT-AVC_V1] CVFI1_SVA_C
> > [JVT-AVC_V1] CVFI2_Sony_H
> > [JVT-AVC_V1] CVFI2_SVA_C
> > [JVT-AVC_V1] CVMA1_Sony_D
> > [JVT-AVC_V1] CVMA1_TOSHIBA_B
> > [JVT-AVC_V1] CVMANL1_TOSHIBA_B
> > [JVT-AVC_V1] CVMANL2_TOSHIBA_B
> > [JVT-AVC_V1] CVMAPAQP3_Sony_E
> > [JVT-AVC_V1] CVMAQP2_Sony_G
> > [JVT-AVC_V1] CVMAQP3_Sony_D
> > [JVT-AVC_V1] CVMP_MOT_FLD_L30_B
> > [JVT-AVC_V1] CVMP_MOT_FRM_L31_B
> > [JVT-AVC_V1] CVNLFI1_Sony_C
> > [JVT-AVC_V1] CVNLFI2_Sony_H
> > [JVT-AVC_V1] CVPA1_TOSHIBA_B
> > [JVT-AVC_V1] FI1_Sony_E
> > [JVT-AVC_V1] MR6_BT_B
> > [JVT-AVC_V1] MR7_BT_B
> > [JVT-AVC_V1] MR8_BT_B
> > [JVT-AVC_V1] MR9_BT_B
> > [JVT-AVC_V1] Sharp_MP_Field_1_B
> > [JVT-AVC_V1] Sharp_MP_Field_2_B
> > [JVT-AVC_V1] Sharp_MP_Field_3_B
> > [JVT-AVC_V1] Sharp_MP_PAFF_1r2
> > [JVT-AVC_V1] Sharp_MP_PAFF_2r
> >