linux-kernel - Re: [EXT] Re: [PATCH v13 08/13] media: amphion: add v4l2 m2m vpu decoder stateful driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <7e63c1665c96feccadd9c05dc603349b3a93900d.camel@ndufresne.ca>
Date:   Fri, 03 Dec 2021 10:09:53 -0500
From:   Nicolas Dufresne <nicolas@...fresne.ca>
To:     Ming Qian <ming.qian@....com>,
        "mchehab@...nel.org" <mchehab@...nel.org>,
        "shawnguo@...nel.org" <shawnguo@...nel.org>,
        "robh+dt@...nel.org" <robh+dt@...nel.org>,
        "s.hauer@...gutronix.de" <s.hauer@...gutronix.de>
Cc:     "hverkuil-cisco@...all.nl" <hverkuil-cisco@...all.nl>,
        "kernel@...gutronix.de" <kernel@...gutronix.de>,
        "festevam@...il.com" <festevam@...il.com>,
        dl-linux-imx <linux-imx@....com>,
        Aisheng Dong <aisheng.dong@....com>,
        "linux-media@...r.kernel.org" <linux-media@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>
Subject: Re: [EXT] Re: [PATCH v13 08/13] media: amphion: add v4l2 m2m vpu
 decoder stateful driver

Le vendredi 03 décembre 2021 à 06:01 +0000, Ming Qian a écrit :
> > -----Original Message-----
> > From: Ming Qian
> > Sent: Friday, December 3, 2021 1:43 PM
> > To: Nicolas Dufresne <nicolas@...fresne.ca>; mchehab@...nel.org;
> > shawnguo@...nel.org; robh+dt@...nel.org; s.hauer@...gutronix.de
> > Cc: hverkuil-cisco@...all.nl; kernel@...gutronix.de; festevam@...il.com;
> > dl-linux-imx <linux-imx@....com>; Aisheng Dong <aisheng.dong@....com>;
> > linux-media@...r.kernel.org; linux-kernel@...r.kernel.org;
> > devicetree@...r.kernel.org; linux-arm-kernel@...ts.infradead.org
> > Subject: RE: [EXT] Re: [PATCH v13 08/13] media: amphion: add v4l2 m2m vpu
> > decoder stateful driver
> > 
> > > -----Original Message-----
> > > From: Nicolas Dufresne [mailto:nicolas@...fresne.ca]
> > > Sent: Friday, December 3, 2021 12:56 PM
> > > To: Ming Qian <ming.qian@....com>; mchehab@...nel.org;
> > > shawnguo@...nel.org; robh+dt@...nel.org; s.hauer@...gutronix.de
> > > Cc: hverkuil-cisco@...all.nl; kernel@...gutronix.de;
> > > festevam@...il.com; dl-linux-imx <linux-imx@....com>; Aisheng Dong
> > > <aisheng.dong@....com>; linux-media@...r.kernel.org;
> > > linux-kernel@...r.kernel.org; devicetree@...r.kernel.org;
> > > linux-arm-kernel@...ts.infradead.org
> > > Subject: [EXT] Re: [PATCH v13 08/13] media: amphion: add v4l2 m2m vpu
> > > decoder stateful driver
> > > 
> > > Caution: EXT Email
> > > 
> > > Le mardi 30 novembre 2021 à 17:48 +0800, Ming Qian a écrit :
> > > > This consists of video decoder implementation plus decoder controls.
> > > > 
> > > > Signed-off-by: Ming Qian <ming.qian@....com>
> > > > Signed-off-by: Shijie Qin <shijie.qin@....com>
> > > > Signed-off-by: Zhou Peng <eagle.zhou@....com>
> > > > ---
> > > >  drivers/media/platform/amphion/vdec.c | 1680
> > > +++++++++++++++++++++++++
> > 
> > 
> > > > +
> > > > +static void vdec_init_fmt(struct vpu_inst *inst) {
> > > > +     struct vdec_t *vdec = inst->priv;
> > > > +     const struct vpu_format *fmt;
> > > > +     int i;
> > > > +
> > > > +     fmt = vpu_helper_find_format(inst, inst->cap_format.type,
> > > vdec->codec_info.pixfmt);
> > > > +     inst->out_format.width = vdec->codec_info.width;
> > > > +     inst->out_format.height = vdec->codec_info.height;
> > > > +     inst->cap_format.width = vdec->codec_info.decoded_width;
> > > > +     inst->cap_format.height = vdec->codec_info.decoded_height;
> > > > +     inst->cap_format.pixfmt = vdec->codec_info.pixfmt;
> > > > +     if (fmt) {
> > > > +             inst->cap_format.num_planes = fmt->num_planes;
> > > > +             inst->cap_format.flags = fmt->flags;
> > > > +     }
> > > > +     for (i = 0; i < inst->cap_format.num_planes; i++) {
> > > > +             inst->cap_format.bytesperline[i] =
> > > vdec->codec_info.bytesperline[i];
> > > > +             inst->cap_format.sizeimage[i] =
> > > vdec->codec_info.sizeimage[i];
> > > > +     }
> > > > +     if (vdec->codec_info.progressive)
> > > > +             inst->cap_format.field = V4L2_FIELD_NONE;
> > > > +     else
> > > > +             inst->cap_format.field = V4L2_FIELD_INTERLACED;
> > > 
> > > As a followup, this should be conditional to the chosen pixel format.
> > > If I understood correct, you produce interlaced is only produce for
> > > linear NV12, for tiled the fields are outputed seperated in their
> > > respective v4l2_buffer. Note sure where yet, but the V4L2 spec
> > > requires you to pair the fields by using the same seq_num on both.
> > 
> > The amphion vpu will store the two fields into one v4l2_buf, So I'll change
> > V4L2_FIELD_INTERLACED to V4L2_FIELD_SEQ_TB
> > 
> 
> Hi Nicolas,
>     Seems gstreamer doesn't support V4L2_FIELD_SEQ_TB yet.
> 
>   switch (fmt.fmt.pix.field) {
>     case V4L2_FIELD_ANY:
>     case V4L2_FIELD_NONE:
>       interlace_mode = GST_VIDEO_INTERLACE_MODE_PROGRESSIVE;
>       break;
>     case V4L2_FIELD_INTERLACED:
>     case V4L2_FIELD_INTERLACED_TB:
>     case V4L2_FIELD_INTERLACED_BT:
>       interlace_mode = GST_VIDEO_INTERLACE_MODE_INTERLEAVED;
>       break;
>     case V4L2_FIELD_ALTERNATE:
>       interlace_mode = GST_VIDEO_INTERLACE_MODE_ALTERNATE;
>       break;
>     default:
>       goto unsupported_field;
>   }

This is correct, I had never had the chance to implement it. So far I only know
IMX6 camera pipeline producing that, but rarely used in practice. What matters
here is that your driver does report the right information so that userspace
don't get fooled into thinking it's interleaved.

> 
> > > 
> > > > +     if (vdec->codec_info.color_primaries ==
> > V4L2_COLORSPACE_DEFAULT)
> > > > +             vdec->codec_info.color_primaries =
> > > V4L2_COLORSPACE_REC709;
> > > > +     if (vdec->codec_info.transfer_chars == V4L2_XFER_FUNC_DEFAULT)
> > > > +             vdec->codec_info.transfer_chars = V4L2_XFER_FUNC_709;
> > > > +     if (vdec->codec_info.matrix_coeffs == V4L2_YCBCR_ENC_DEFAULT)
> > > > +             vdec->codec_info.matrix_coeffs = V4L2_YCBCR_ENC_709;
> > > > +     if (vdec->codec_info.full_range == V4L2_QUANTIZATION_DEFAULT)
> > > > +             vdec->codec_info.full_range =
> > > V4L2_QUANTIZATION_LIM_RANGE;
> > > > +}
> > > > +