linux-kernel - Re: [PATCH v2 1/2] media: docs-rst: Document memory-to-memory video decoder interface

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3ea3bf5bf9904ce877142c41f595207752172d27.camel@ndufresne.ca>
Date:   Thu, 24 Jan 2019 14:55:16 -0500
From:   Nicolas Dufresne <nicolas@...fresne.ca>
To:     Tomasz Figa <tfiga@...omium.org>, Hans Verkuil <hverkuil@...all.nl>
Cc:     Linux Media Mailing List <linux-media@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Pawel Osciak <posciak@...omium.org>,
        Alexandre Courbot <acourbot@...omium.org>,
        Kamil Debski <kamil@...as.org>,
        Andrzej Hajda <a.hajda@...sung.com>,
        Kyungmin Park <kyungmin.park@...sung.com>,
        Jeongtae Park <jtp.park@...sung.com>,
        Philipp Zabel <p.zabel@...gutronix.de>,
        Tiffany Lin (林慧珊) 
        <tiffany.lin@...iatek.com>,
        Andrew-CT Chen (陳智迪) 
        <andrew-ct.chen@...iatek.com>,
        Stanimir Varbanov <stanimir.varbanov@...aro.org>,
        Todor Tomov <todor.tomov@...aro.org>,
        Paul Kocialkowski <paul.kocialkowski@...tlin.com>,
        Laurent Pinchart <laurent.pinchart@...asonboard.com>,
        dave.stevenson@...pberrypi.org,
        Ezequiel Garcia <ezequiel@...labora.com>,
        Maxime Jourdan <maxi.jourdan@...adoo.fr>
Subject: Re: [PATCH v2 1/2] media: docs-rst: Document memory-to-memory video
 decoder interface

Le jeudi 24 janvier 2019 à 18:06 +0900, Tomasz Figa a écrit :
> > Actually I just realized the last point might not even be achievable
> > for some of the decoders (s5p-mfc, mtk-vcodec), as they don't report
> > which frame originates from which bitstream buffer and the driver just
> > picks the most recently consumed OUTPUT buffer to copy the timestamp
> > from. (s5p-mfc actually "forgets" to set the timestamp in some cases
> > too...)
> > 
> > I need to think a bit more about this.
> 
> Actually I misread the code. Both s5p-mfc and mtk-vcodec seem to
> correctly match the buffers.

Ok good, since otherwise it would have been a regression in MFC driver.
This timestamp passing thing could in theory be made optional though,
it lives under some COPY_TIMESTAMP kind of flag. What that means though
is that a driver without such a capability would need to signal dropped
frames using some other mean.

In userspace, the main use is to match the produced frame against a
userspace specific list of frames. At least this seems to be the case
in Gst and Chromium, since the userspace list contains a superset of
the metadata found in the v4l2_buffer.

Now, using the produced timestamp, userspace can deduce frame that the
driver should have produced but didn't (could be a deadline case codec,
or simply the frames where corrupted). It's quite normal for a codec to
just keep parsing until it finally find something it can decode.

That's at least one way to do it, but there is other possible
mechanism. The sequence number could be used, or even producing buffers
with the ERROR flag set. What matters is just to give userspace a way
to clear these frames, which would simply grow userspace memory usage
over time.

Nicolas