linux-kernel - [PATCH 0/9] Enable two H264 encoder cores on MT8195

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20210816105934.28265-1-irui.wang@mediatek.com>
Date:   Mon, 16 Aug 2021 18:59:25 +0800
From:   Irui Wang <irui.wang@...iatek.com>
To:     Hans Verkuil <hverkuil-cisco@...all.nl>,
        Tzung-Bi Shih <tzungbi@...omium.org>,
        Alexandre Courbot <acourbot@...omium.org>,
        Tiffany Lin <tiffany.lin@...iatek.com>,
        Andrew-CT Chen <andrew-ct.chen@...iatek.com>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Rob Herring <robh+dt@...nel.org>,
        Matthias Brugger <matthias.bgg@...il.com>,
        Tomasz Figa <tfiga@...gle.com>, Yong Wu <yong.wu@...iatek.com>
CC:     Hsin-Yi Wang <hsinyi@...omium.org>,
        Maoguang Meng <maoguang.meng@...iatek.com>,
        Longfei Wang <longfei.wang@...iatek.com>,
        Yunfei Dong <yunfei.dong@...iatek.com>,
        Fritz Koenig <frkoenig@...omium.org>,
        Irui Wang <irui.wang@...iatek.com>,
        <linux-media@...r.kernel.org>, <devicetree@...r.kernel.org>,
        <linux-kernel@...r.kernel.org>,
        <linux-arm-kernel@...ts.infradead.org>,
        <srv_heupstream@...iatek.com>,
        <linux-mediatek@...ts.infradead.org>,
        <Project_Global_Chrome_Upstream_Group@...iatek.com>
Subject: [PATCH 0/9] Enable two H264 encoder cores on MT8195

MT8195 has two H264 encoder cores, they have their own power-domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance.

This series of patches is to use enable two h264 encoder cores.
path[1..2]: use linux component framework to manage encoder hardware,
user call "mt8195-vcodec-enc" driver can get the encoder master device,
the encoding work is done by the two encoder core device. The hw_mode
variable is added to distinguish from old platform, two encoder cores
called "FRAME_RACING_MODE".

The hardware mode of two encoder cores work together(overlap, another
word called) on MT8195 called "frame_racing_mode", the two encoder
power-domains should be power on together while encoding, the encoding
process look like this:

    VENC Core0 frm#0....frm#2....frm#4
    VENC Core1  .frm#1....frm#3....frm#5

patch[3..5]: due to the component device, the master device has no
power-domains/clocks properties in dtsi, so the power/clock init function
can't use for "frame_racing_mode" device in master device probe process,
it should be called in component device probe process. Power on the
hardware power and clock on demand.

patch[6]: "frame_racing_mode" encoding need a new set of memory buffer
for two encoder cores. For compatibility, we should new a encoder driver
interface.

patch[7..9]: add "frame_racing_mode" encoding process:
As-Is: Synchronous
VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish

VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encode done with result --> job_finish
...

To-Be: Asynchronous
VIDIOC_QBUF#0 --> device_run(triger encoder core0) --> job_finish
..VIDIOC_QBUF#1 --> device_run(triger encoder core1) --> job_finish
(core0 may encode done here, return encode result to client)
VIDIOC_QBUF#2 --> device_run(triger encoder core0) --> job_finish

Thers is no "wait encoder IRQ" synchronous call during "frame_racing_mode"
encoding process, which can full use the two encoder cores to achieve
higher performance.

Irui Wang (9):
  dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores
  media: mtk-vcodec: Use component framework to manage encoder hardware
  media: mtk-vcodec: Rewrite venc power manage interface
  media: mtk-vcodec: Add venc power on/off interface
  media: mtk-vcodec: Rewrite venc clock interface
  media: mtk-vcodec: Add new venc drv interface for frame_racing mode
  media: mtk-vcodec: Add frame racing mode encode process
  media: mtk-vcodec: Return encode result to client
  media: mtk-vcodec: Add delayed worker for encode timeout

 .../bindings/media/mediatek-vcodec.txt        |   2 +
 drivers/media/platform/mtk-vcodec/Makefile    |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  34 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 120 +++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  10 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 204 +++++-
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.c   | 253 +++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_hw.h   |  38 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 213 ++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  13 +-
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 +
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
 .../platform/mtk-vcodec/venc/venc_common_if.c | 675 ++++++++++++++++++
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |   6 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   2 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  96 ++-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
 19 files changed, 1564 insertions(+), 149 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_hw.h
 create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c

-- 
2.25.1