| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <67e956df4cc0b2b9609e5f9366aff9b42bffc75e.camel@mediatek.com>
Date: Mon, 26 May 2025 02:13:13 +0000
From: CK Hu (胡俊光) <ck.hu@...iatek.com>
To: "robh@...nel.org" <robh@...nel.org>, "mchehab@...nel.org"
<mchehab@...nel.org>, AngeloGioacchino Del Regno
<angelogioacchino.delregno@...labora.com>, "matthias.bgg@...il.com"
<matthias.bgg@...il.com>, Olivia Wen (温倩苓)
<Olivia.Wen@...iatek.com>
CC: "linux-media@...r.kernel.org" <linux-media@...r.kernel.org>,
Project_Global_Chrome_Upstream_Group
<Project_Global_Chrome_Upstream_Group@...iatek.com>,
"devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"krzk+dt@...nel.org" <krzk+dt@...nel.org>,
"linux-arm-kernel@...ts.infradead.org"
<linux-arm-kernel@...ts.infradead.org>,
Teddy Chen (陳乾元) <Teddy.Chen@...iatek.com>,
"conor+dt@...nel.org" <conor+dt@...nel.org>,
"linux-mediatek@...ts.infradead.org" <linux-mediatek@...ts.infradead.org>,
"yunkec@...omium.org" <yunkec@...omium.org>
Subject: Re: [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API
On Sat, 2025-05-24 at 19:49 +0800, Olivia Wen wrote:
> Add controls and formats for MediaTek Imgsys processing
>
> Signed-off-by: Olivia Wen <olivia.wen@...iatek.com>
> ---
> include/uapi/linux/mtkisp_imgsys.h | 215 +++++++++++++++++++++++++++++
> 1 file changed, 215 insertions(+)
> create mode 100644 include/uapi/linux/mtkisp_imgsys.h
>
> diff --git a/include/uapi/linux/mtkisp_imgsys.h b/include/uapi/linux/mtkisp_imgsys.h
> new file mode 100644
> index 000000000000..c7528c8bf4a2
> --- /dev/null
> +++ b/include/uapi/linux/mtkisp_imgsys.h
> @@ -0,0 +1,215 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * MediaTek ISP imgsys User space API
> + *
> + * Copyright (c) 2024 MediaTek Inc.
> + */
> +
> +#ifndef _MTKISP_IMGSYS_USER_H
> +#define _MTKISP_IMGSYS_USER_H
> +
> +#include <linux/v4l2-controls.h>
> +
> +/******************************************************/
> +/* MediaTek ISP imgsys controls */
> +/******************************************************/
> +
> +/* The base for the MediaTek imgsys driver controls */
> +/* We reserve 48 controls for this driver. */
> +#define V4L2_CID_USER_MTK_IMG_BASE (V4L2_CID_USER_BASE + 0x1100)
> +
> +/**
> + * Define a control identifier, V4L2_CID_MTK_IMG_RESIZE_RATIO, for setting the
> + * image processing resize ratio for MediaTek chip. This identifier allows
> + * adjustment of the image resize ratio to accommodate various display or
> + * processing needs. ImgSys hardware supports multiple resizers, including
> + * any ratio, downscaling by factors of 2, 4, and 42, with NULL indicating
> + * the end of menu options.
> + */
> +#define V4L2_CID_MTK_IMG_RESIZE_RATIO (V4L2_CID_USER_MTK_IMG_BASE + 34)
> +
> +/******************************************************/
> +/* Vendor specific - MediaTek ISP formats */
> +/******************************************************/
> +
> +/* MediaTek warp map 32-bit, 2 plane */
> +#define V4L2_PIX_FMT_WARP2P v4l2_fourcc('M', 'W', '2', 'P')
> +/* YUV-10bit packed 4:2:0 2plane, (Y)(UV) */
Pixel format should define in include/uapi/linux/videodev2.h
Add document to describe the detail of V4L2_PIX_FMT_WARP2P. [1] is the example document for V4L2_META_FMT_RPI_FE_CFG.
There are many document in Documentation/userspace-api/media/v4l/ for your reference.
[1] https://patchwork.kernel.org/project/linux-media/patch/20241003-rp1-cfe-v6-1-d6762edd98a8@ideasonboard.com/
> +#define V4L2_PIX_FMT_YUV_2P010P v4l2_fourcc('U', '0', '2', 'A')
> +/* YUV-12bit packed 4:2:0 2plane, (Y)(UV) */
Ditto.
> +#define V4L2_PIX_FMT_YUV_2P012P v4l2_fourcc('U', '0', '2', 'C')
> +/* Y-32bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y32 v4l2_fourcc('M', 'T', '3', '2')
> +/* Y-16bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y16 v4l2_fourcc('M', 'T', '1', '6')
> +/* Y-8bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y8 v4l2_fourcc('M', 'T', '0', '8')
> +
> +/**
> + * Describes the MediaTek APLY8(V4L2_PIX_FMT_MTISP_APLY8) format, which includes
> + * Y-8bit image data and appends APL(Average Pixel Level) data utilized by the
> + * ME(motion estimation) hardware to the end of the image.
> + *
> + * The hardware ME generates motion vectors by processing two consecutive frames,
> + * referred to as the previous and current frames, to enhance graphic processing.
> + * The diagram is shown below.
> + *
> + * +------------------+
> + * the thumbnail of previous frame -> | |
> + * (include the L0 frame and L1 frame)| |
> + * | ME | -> motion vectors
> + * the thumbnail of current frame -> | |
> + * (include the L0 frame and L1 frame)| |
> + * +------------------+
> + *
> + * To accelerate computation, the two images will be resized into two different
> + * smaller images, referred to as L0 frame, L1 frame and APL data.
> + * The L1 frame is a scaled-down version of the L0 frame, reduced to one-fourth
> + * of its original size containing APL data.
> + *
> + * +------------------+
> + * previous L0 frame -> | |
> + * previous L1 frame -> | |
> + * (include APL data) | |
> + * | ME | -> motion vectors
> + * current L0 frame -> | |
> + * current L1 frame -> | |
> + * (include APL data) | |
> + * +------------------+
> + *
> + * Each frame size:
> + * the width of L0 frame (L0_wd) = 576 (fixed value)
> + * the height of L0 frame (L0_ht) =
> + * 16 pixels align((the width of original image * L0_wd) / the height of original image)
> + * the width of L1 frame (L1_wd) = L0_wd / 4
> + * the height of L1 frame (L1_ht) = L0_ht / 4
> + *
> + * Buffer size:
> + * line_size = width * bytes_per_pixel(1)
> + * stride with 16 bytes alignment = ((line_size + 15) / 16) * 16
> + * buffer_size = stride with 16 bytes alignment * height + APL data(16 bytes)
> + *
> + * And the memory layout should be followed as
> + *
> + * @code
> + * APLY8
> + * +------------------+
> + * | Y Plane |
> + * | Bit Stream |
> + * +------------------+
> + * | APL data |
> + * +------------------+
> + * @endcode
> + *
> + * @note The APL data size is 16 bytes.The caller SHOULD NOT modify it.
> + */
> +/* Y-8bit image data and append the APL data */
> +#define V4L2_PIX_FMT_MTISP_APLY8 v4l2_fourcc('M', 'T', 'A', '8')
Ditto.
> +
> +/* Packed 10-bit */
> +#define V4L2_PIX_FMT_MTISP_SBGGR10 v4l2_fourcc('M', 'B', 'B', 'A')
> +#define V4L2_PIX_FMT_MTISP_SGBRG10 v4l2_fourcc('M', 'B', 'G', 'A')
> +#define V4L2_PIX_FMT_MTISP_SGRBG10 v4l2_fourcc('M', 'B', 'g', 'A')
> +#define V4L2_PIX_FMT_MTISP_SRGGB10 v4l2_fourcc('M', 'B', 'R', 'A')
Ditto.
> +
> +/* Vendor specific - MediaTek ISP parameters for firmware */
> +/* ISP tuning parameters */
> +#define V4L2_META_FMT_MTISP_TUN_PARAMS v4l2_fourcc('M', 'T', 'f', 't')
> +/* ISP statistics parameters */
> +#define V4L2_META_FMT_MTISP_STAT_PARAMS v4l2_fourcc('M', 'T', 'f', 's')
> +/* ISP controlling parameters */
> +#define V4L2_META_FMT_MTISP_CTL_PARAMS v4l2_fourcc('M', 'T', 'f', 'c')
Ditto.
> +
> +/*****************************************************************************/
> +/* Define sizes for various formats used in the MediaTek image system driver */
> +/*****************************************************************************/
> +/**
> + * SIZE_OF_APL_DATA - Size of the APL (Average Pixel Level) data
> + *
> + * This constant defines the size of the APL data, which is 16 bytes.
> + * The APL data is appended to the image format defined by
> + * V4L2_PIX_FMT_MTISP_APLY8. This format includes 8-bit Y image data
> + * followed by the APL data, which is utilized by the motion estimation
> + * hardware for enhanced processing. The APL data provides important
> + * information about the average pixel levels, which can be critical
> + * for various image processing tasks.
> + */
> +#define SIZE_OF_APL_DATA 16 /* Size of APL data used by V4L2_PIX_FMT_MTISP_APLY8 */
> +
> +/**
> + * MediaTek Tuning Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_TUN_PARAMS.
> + * This buffer contains essential tuning information for the MediaTek camera,
> + * including:
> + * - Sensor tuning settings
Ditto.
> + * - Digital gain for adjusting image brightness
Ditto.
> + * - White balance gain
Ditto.
> + *
> + * Note: The individual fields of the tuning metadata structure are not exposed
> + * to the kernel. The kernel only requires the size of the tuning metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the tuning metadata without impacting the
> + * kernel's interface.
> + */
> + #define SIZE_OF_TUNING_META 219352
> +
> +/**
> + * MediaTek Control Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_CTL_PARAMS.
> + * The control metadata buffer aggregates control information for multiple
> + * image processing modules, including TRAW, DIP, PQDIP, ME, WPE, and ADL.
> + * It facilitates the configuration and management of these modules by
> + * encapsulating relevant parameters and settings required for processing
> + * image data efficiently. This structure serves as a central point for
> + * coordinating the operation of different hardware components in the image
> + * processing pipeline.
> + *
> + * Note: The individual fields of the control metadata structure are not exposed
> + * to the kernel. The kernel only requires the size of the control metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the control metadata without impacting the
> + * kernel's interface.
How user space to generate these control meta data?
Add document about how generating these control meta data.
> + */
> +#define SIZE_OF_COMMON_CTRL 688
> +#define SIZE_OF_WPE_CTRL 240
> +#define SIZE_OF_TRAW_CTRL 120
> +#define SIZE_OF_DIP_CTRL 112
> +#define SIZE_OF_PQDIP_CTRL 112
> +#define SIZE_OF_ME_CTRL 8
> +#define SIZE_OF_ADL_CTRL 64
> +#define SIZE_OF_YUFO_META_INFO_WPE 4228
> +#define SIZE_OF_YUFO_META_INFO_DIP 4228
> +
> +#define SIZE_OF_CTRL_META ( \
> + SIZE_OF_COMMON_CTRL + \
> + SIZE_OF_WPE_CTRL + \
> + SIZE_OF_TRAW_CTRL + \
> + SIZE_OF_DIP_CTRL + \
> + SIZE_OF_PQDIP_CTRL + \
> + SIZE_OF_ME_CTRL + \
> + SIZE_OF_ADL_CTRL + \
> + SIZE_OF_YUFO_META_INFO_WPE + \
> + SIZE_OF_YUFO_META_INFO_DIP \
> +)
> +
> +/**
> + * MediaTek Statistics Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_STAT_PARAMS.
> + * This buffer contains essential statistics information for the MediaTek camera,
> + * including:
> + * - Readiness and corresponding values for tone and contrast statistics
> + * - Motion estimation (ME) statistics
> + * - ME feature match blocks statistics
> + *
> + * Note: The individual fields of the statistics buffer structure are not exposed
> + * to the kernel. The kernel only requires the size of the statistics metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the statistics metadata without impacting the
> + * kernel's interface.
How user space to generate these statistics meta data?
Add document about how generating these statistics meta data.
Regards,
CK
> + */
> + #define SIZE_OF_STATISTICS_META 88
> +
> +#endif /* _MTKISP_IMGSYS_USER_H */
Powered by blists - more mailing lists