linux-kernel - Re: [PATCH 3/5] media: dt-bindings: Add Apple ISP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250219115737.GB26386@robin.jannau.net>
Date: Wed, 19 Feb 2025 12:57:37 +0100
From: Janne Grunau <j@...nau.net>
To: Laurent Pinchart <laurent.pinchart@...asonboard.com>
Cc: Sasha Finkelstein <fnkl.kernel@...il.com>,
	Krzysztof Kozlowski <krzk@...nel.org>,
	Sven Peter <sven@...npeter.dev>,
	Alyssa Rosenzweig <alyssa@...enzweig.io>,
	Rob Herring <robh@...nel.org>,
	Krzysztof Kozlowski <krzk+dt@...nel.org>,
	Conor Dooley <conor+dt@...nel.org>,
	Hector Martin <marcan@...can.st>,
	Ulf Hansson <ulf.hansson@...aro.org>,
	Mauro Carvalho Chehab <mchehab@...nel.org>,
	Shawn Guo <shawnguo@...nel.org>,
	Sascha Hauer <s.hauer@...gutronix.de>,
	Pengutronix Kernel Team <kernel@...gutronix.de>,
	Fabio Estevam <festevam@...il.com>, asahi@...ts.linux.dev,
	linux-arm-kernel@...ts.infradead.org, devicetree@...r.kernel.org,
	linux-kernel@...r.kernel.org, linux-pm@...r.kernel.org,
	linux-media@...r.kernel.org, imx@...ts.linux.dev
Subject: Re: [PATCH 3/5] media: dt-bindings: Add Apple ISP

On Wed, Feb 19, 2025 at 12:53:26PM +0200, Laurent Pinchart wrote:
> On Wed, Feb 19, 2025 at 10:54:31AM +0100, Sasha Finkelstein wrote:
> > On Wed, 19 Feb 2025 at 10:37, Krzysztof Kozlowski <krzk@...nel.org> wrote:
> > > > +
> > > > +  apple,platform-id:
> > > > +    description: Platform id for firmware
> > > > +    $ref: /schemas/types.yaml#/definitions/uint32
> > >
> > >
> > > No, use firmware-name.
> > 
> > Not sure how is firmware-name an appropriate field, fw-name is a string
> > that references a firmware file, while this field is an id that is sent to the
> > coprocessor firmware in order to identify the platform.
> > 
> > > > +  apple,temporal-filter:
> > > > +    description: Whether temporal filter should be enabled in firmware
> > > > +    $ref: /schemas/types.yaml#/definitions/uint32
> > >
> > > And why is this not enabled always? Why this is board specific?
> > 
> > Not every board has support for this feature.
> > 
> > > You miss here ports or port. ISP usually gets signal from some camera or
> > > other block.
> > 
> > For complex cameras - yes, but this is closer to a UVC camera connected
> > via a bespoke protocol. We do not need to deal with the sensor access,
> > all of it is managed by the coprocessor firmware.
> > 
> > > > +        properties:
> > > > +          apple,config-index:
> > > > +            description: Firmware config index
> > > > +            $ref: /schemas/types.yaml#/definitions/uint32
> > >
> > >
> > > No duplicated indices. You have reg for this, assuming this is index.
> > 
> > There are duplicated indices, see isp-imx248.dtsi in patch 5 for an example.
> > 
> > > All these do not look like hardware properties but rather configuration
> > > of sensor which should be done runtime by OS, not by DT.
> > 
> > Those are board-specific and not discoverable via the ISP protocol.
> 
> But they are settable through the ISP protocol, aren't they ? For
> instance, looking at isp-imx248.dtsi, the first four entries are
> 
> 	/* 1280x720 */
> 	preset0 {
> 		apple,config-index = <0>;
> 		apple,input-size = <1296 736>;
> 		apple,output-size = <1280 720>;
> 		apple,crop = <8 8 1280 720>;
> 	};
> 
> 	/* 960x720 (4:3) */
> 	preset1 {
> 		apple,config-index = <0>;
> 		apple,input-size = <1296 736>;
> 		apple,output-size = <960 720>;
> 		apple,crop = <168 8 960 720>;
> 	};
> 
> 	/* 960x540 (16:9) */
> 	preset2 {
> 		apple,config-index = <0>;
> 		apple,input-size = <1296 736>;
> 		apple,output-size = <960 540>;
> 		apple,crop = <8 8 1280 720>;
> 	};
> 
> 	/* 640x480 (4:3) */
> 	preset3 {
> 		apple,config-index = <0>;
> 		apple,input-size = <1296 736>;
> 		apple,output-size = <640 480>;
> 		apple,crop = <168 8 960 720>;
> 	};
> 
> But I may be interested in capturing a 640x480 frame with cropping only
> and without scaling, with
> 
> input-size = 1296x736
> output-size = 640x480
> crop = (328,128)/640x480
> 
> Or I may want my cropped frame to be located in the upper-left corner:
> 
> input-size = 1296x736
> output-size = 640x480
> crop = (8,8)/640x480
> 
> If I set those parameters through the ISP protocol, won't it work ?

If my memory serves me right the presets wre added as workaround for
userspace not handling V4L2_FRMSIZE_TYPE_STEPWISE well (or at all) and
the added complexity of handling the qadratic sensor with partially
occluded or outside of the usable lens diameter corners.

It is a simplified description of the hardware to make it useable for
most software which is expected simple uvc cameras.

There are still two common issues in user space software related to this
driver:
- software expects width == linesize
- resolution selection is based frame height, i.e. it prefers 1080x1920
  over 1920x1080 on devices with quadratic sensor.

ciao Janne