linux-kernel - Re: [PATCH] media: hantro: Hook up RK3399 JPEG encoder output

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ae2fa5b455fce8f159e02b3156800b162e424a21.camel@ndufresne.ca>
Date:   Thu, 25 Nov 2021 10:41:07 -0500
From:   Nicolas Dufresne <nicolas@...fresne.ca>
To:     Hans Verkuil <hverkuil@...all.nl>,
        Chen-Yu Tsai <wenst@...omium.org>
Cc:     Ezequiel Garcia <ezequiel@...guardiasur.com.ar>,
        Philipp Zabel <p.zabel@...gutronix.de>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        linux-media@...r.kernel.org, linux-rockchip@...ts.infradead.org,
        linux-staging@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] media: hantro: Hook up RK3399 JPEG encoder output

Le jeudi 25 novembre 2021 à 09:44 +0100, Hans Verkuil a écrit :
> On 23/11/2021 20:39, Nicolas Dufresne wrote:
> > Le lundi 22 novembre 2021 à 11:57 +0800, Chen-Yu Tsai a écrit :
> > > On Sat, Nov 20, 2021 at 12:00 AM Nicolas Dufresne <nicolas@...fresne.ca> wrote:
> > > > 
> > > > Le vendredi 19 novembre 2021 à 15:46 +0800, Chen-Yu Tsai a écrit :
> > > > > The JPEG encoder found in the Hantro H1 encoder block only produces a
> > > > > raw entropy-encoded scan. The driver is responsible for building a JPEG
> > > > > compliant bitstream and placing the entropy-encoded scan in it. Right
> > > > > now the driver uses a bounce buffer for the hardware to output the raw
> > > > > scan to.
> > > > > 
> > > > > In commit e765dba11ec2 ("hantro: Move hantro_enc_buf_finish to JPEG
> > > > > codec_ops.done"), the code that copies the raw scan from the bounce
> > > > > buffer to the capture buffer was moved, but was only hooked up for the
> > > > > Hantro H1 (then RK3288) variant. The RK3399 variant was broken,
> > > > > producing a JPEG bitstream without the scan, and the capture buffer's
> > > > > .bytesused field unset.
> > > > > 
> > > > > Fix this by duplicating the code that is executed when the JPEG encoder
> > > > > finishes encoding a frame. As the encoded length is read back from
> > > > > hardware, and the variants having different register layouts, the
> > > > > code is duplicated rather than shared.
> > > > > 
> > > > > Fixes: e765dba11ec2 ("hantro: Move hantro_enc_buf_finish to JPEG codec_ops.done")
> > > > > Signed-off-by: Chen-Yu Tsai <wenst@...omium.org>
> > > > > ---
> > > > > This was developed on the downstream ChromeOS 5.10 kernel (with a hack
> > > > > for .data_offset) and tested with ChromeOS's jpeg_encode_accelerator_unittest
> > > > > patched to accept non-JFIF JPEG streams (https://crrev.com/c/3291480).
> > > > > 
> > > > > This was then forward-ported to mainline (name and filename changes) and
> > > > > compile tested only.
> > > > 
> > > > Tested with GStreamer on top of 5.16-rc1 from media_stage.git. Not perfect but
> > > > at least the the output it valid. Test command was:
> > > > 
> > > >   gst-launch-1.0 videotestsrc num-buffers=2 ! v4l2jpegenc ! filesink
> > > > location=test.jpg
> > > > 
> > > > Notice that I encode two frames, it seems like the draining flow is broken in
> > > > this driver. GStreamer will queue the frame and issue CMD_START immediately, the
> > > > driver will skip the encode, leaving me with an empty file.
> > > 
> > > The hantro driver doesn't implement ENC_CMD, which IIRC is used for the
> > > draining flow. I guess that's something to fix, since the mem2mem stateful
> > > encoder spec seems to require it. Or does that spec not apply to the JPEG
> > > encoders?
> > 
> > I'm pretty sure its required. But this isn't a regression from this patch.
> 
> I don't think it is required for JPEG encoders today. Each frame is independent of
> any other, so it behaves just like a m2m scaler for example.
> 
> It doesn't hurt to support it, but it shouldn't be necessary for jpeg codecs.

I believe the requirement in the spec is deliberate to avoid making userland
work a night mare. If draining a filled queue requires counting frames on top of
using CMD_STOP, you are duplicating the effort.

I have a strong opinion about this, I believe consistency is key, the frame work
should provide this feature for JPEG decoder/encoder trivially imho and until
then, driver should all support the same draining flow. I'm not going to relax
this in GStreamer.

Nicolas

> 
> If there is a reason why this is needed, then it would likely also be needed for
> m2m devices like scalers.
> 
> Regards,
> 
> 	Hans
> 
> > 
> > > 
> > > > Tested-by: Nicolas Dufresne <nicolas.dufresne@...labora.com>
> > > 
> > > Thanks!
> > > 
> > > ChenYu
> > 
>