linux-kernel - Re: [PATCH 2/2] [WIP]: media: Add Synaptics compressed tiled format

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ede840d10677041d2c8d0a07b81b3cd55a6c92ba.camel@ndufresne.ca>
Date:   Tue, 23 Aug 2022 09:44:22 -0400
From:   Nicolas Dufresne <nicolas@...fresne.ca>
To:     Hsia-Jun Li <Randy.Li@...aptics.com>
Cc:     Tomasz Figa <tfiga@...omium.org>, dri-devel@...ts.freedesktop.org,
        maarten.lankhorst@...ux.intel.com, mripard@...nel.org,
        tzimmermann@...e.de, airlied@...ux.ie, daniel@...ll.ch,
        mchehab@...nel.org, hverkuil-cisco@...all.nl,
        ezequiel@...guardiasur.com.ar, sakari.ailus@...ux.intel.com,
        ribalda@...omium.org,
        Laurent Pinchart <laurent.pinchart@...asonboard.com>,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        sebastian.hesselbarth@...il.com, jszhang@...nel.org,
        linux-arm-kernel@...ts.infradead.org
Subject: Re: [PATCH 2/2] [WIP]: media: Add Synaptics compressed tiled format

Le mardi 23 août 2022 à 15:40 +0800, Hsia-Jun Li a écrit :
> > In current state, If your driver can support it, userland does not strictly
> > need
> > to re-allocate if the resolution is changed to smaller. In most SVC
> > scenarios,
> > the largest resolution is known in advance, so pre-allocation can happen to
> > the
> When you play a video from Youtube, you may notice that starting 
> resolution is low, then after it received more data knowning the 
> bandwidth is enough, it would switch to a higher resolution. I don't 
> think it would inform the codecs2 or OMX there is a higher target 
> resolution.
> 
> Besides, for the case of SVC in a conference system, the remote(gatway) 
> would not tell you there is a higer resolution or frame rate because you 
> can't receive it in negotiate stage, it could be permanently(device 
> capability) or just bandwidth problem. Whether we know there is a higher 
> requirement video depends on the transport protocols used here.
> 
> The basic idea of SVC is that the low layer didn't depends on the upper 
> layer, we can't tell how the bitstream usually.

I'm not saying against the fact the for drivers without IOMMU (hitting directly
into the CMA allocator), allocation latency is massive challenge, and a
mechanism to smoothly reallocate (rather then mass-reallocation) is needed in
the long run. This is what I'm referring to when saying that folks have
considered extending CREATE_BUFS() with a DELETE_BUFS() ioctl.

Note that there is tones of software trickery you can use to mitigate this. The
most simple one is to use CREATE_BUFS() instead of REQBUFS(). Instead of
reallocating all the buffers you need in one go, you would allocate them one by
one. This will distribute allocation latency. For stateful CODEC, most OMX
focused firmware needs to be modified for that, since they stick with the old
OMX spec which did not allow run-time allocation.

Another trick is to use a second codec session. Both stateful/stateless CODEC
have support for concurrent decoding. On the MSE requirement, is that the stream
transition happens only on keyframe boundary. Meaning, there is no need to reuse
the same session, you can create a new decoder in parallel, and that before the
drain is complete (after the event, before the last buffer). This will compress
the "setup" latency, to the cost of some extra memory usage. Specially in the
MSE case, this is nearly always possible since browsers do require support for
more then 1 concurrent decode. This method also works with OMX style CODEC
without any modification.

regards,
Nicolas