Message-ID: <2ktcdemu3og64vuvyxykpxrwquewubsqcpmhxykynkbl5ddwym@dsxaldpwu2u4>
Date: Wed, 15 Jan 2025 11:35:19 +0100
From: Mehdi Djait <mehdi.djait@...ux.intel.com>
To: Michael Riesch <michael.riesch@...fvision.net>
Cc: Laurent Pinchart <laurent.pinchart@...asonboard.com>, 
	Maxime Chevallier <maxime.chevallier@...tlin.com>, Théo Lebrun <theo.lebrun@...tlin.com>, 
	Thomas Petazzoni <thomas.petazzoni@...tlin.com>, Mauro Carvalho Chehab <mchehab@...nel.org>, 
	Rob Herring <robh+dt@...nel.org>, Krzysztof Kozlowski <krzk+dt@...nel.org>, 
	Conor Dooley <conor+dt@...nel.org>, Heiko Stuebner <heiko@...ech.de>, 
	Kever Yang <kever.yang@...k-chips.com>, Nicolas Dufresne <nicolas@...fresne.ca>, 
	Sebastian Fricke <sebastian.fricke@...labora.com>, Alexander Shiyan <eagle.alexander923@...il.com>, 
	Val Packett <val@...kett.cool>, Rob Herring <robh@...nel.org>, 
	Philipp Zabel <p.zabel@...gutronix.de>, Sakari Ailus <sakari.ailus@...ux.intel.com>, 
	linux-media@...r.kernel.org, devicetree@...r.kernel.org, linux-kernel@...r.kernel.org, 
	linux-arm-kernel@...ts.infradead.org, linux-rockchip@...ts.infradead.org, 
	Mehdi Djait <mehdi.djait@...tlin.com>, Gerald Loacker <gerald.loacker@...fvision.net>
Subject: Re: [PATCH v2 0/6] media: rockchip: add a driver for the rockchip
 camera interface (cif)

Hi Michael and Laurent,

On Fri, Jan 10, 2025 at 09:48:04AM +0100, Michael Riesch wrote:
> Hi Laurent,
> 
> On 1/9/25 18:12, Laurent Pinchart wrote:
> > Hi Michael,
> > 
> > On Tue, Dec 17, 2024 at 04:55:12PM +0100, Michael Riesch wrote:
> >> [...]
> >> and refactor the whole thing. The resulting v2 of the series now adds a
> >> basic media controller centric V4L2 driver for the Rockchip CIF with
> >>  - support for the PX30 VIP (not tested, though, due to the lack of HW)
> >>  - support for the RK3568 VICAP DVP
> >>  - abstraction for the ping-pong scheme to allow for future extensions 
> >>
> >> However, several features are not yet addressed, such as
> >>  - support for the RK3568 MIPI CSI-2 receiver
> > 
> > We've discussed this previously on IRC, but I don't think the issue has
> > been raised on the list.
> > 
> > I'm puzzled by how this will work. As far as I understand, the RK3568
> > has a CSI-2 receiver with 4 data lanes and 2 clock lanes. I assume this
> > is used to support both 2x2 lanes and 1x4 lanes. Both the VICAP and ISP
> > sections of the TRM list CSI2 RX registers, but it's not clear how the
> > components are all connected. Does the ISP need to be part of the same
> > media graph ?
> 
> Well you are not the only one to be puzzled by this HW :-) I started to
> bring up the MIPI CSI-2 part and can describe the situation as I
> understand it. No claim for correctness or completeness, though.
> 
> Indeed, the RK3568 MIPI CSI-2 PHY (TRM Chapter 28) has 4 data lanes and
> 2 clock lanes. And indeed it is possible to attach one device (1x4) or
> two devices (2x2) to it. In the latter case the PHY is operated in split
> mode (and 1x4 would be the default full mode, then). The mode can be set
> in a GRF register.
> 
> The RK3568 features a MIPI CSI-2 host controller (TRM Chapter 27) that
> (apparently) is connected to the RK3568 VICAP but has its registers in a
> separate memory region. Right now I am trying to clean up the downstream
> V4L2 subdevice driver for this host controller, which shall be linked to
> the PHY using the "phys" DT property, and linked to the VICAP using DT
> endpoints. In the end, the media graph could look like
> 
>   CC1 (V4L2 subdev) --> RK3568 VICAP DVP (V4L2 device)
> 
>   CC2 (V4L2 subdev) --> RK3568 MIPI CSI-2 HOST (V4L2 subdev) -->
>   RK3568 VICAP MIPI (V4L2 device)
> 
> where CC denotes a companion chip, such as an image sensor or a video
> decoder. Note that there are two disjoint subgraphs in this topology.
> 
> As you already pointed out, the RK3568 ISP features its integrated MIPI
> CSI-2 host controller (as e.g. the RK3399 ISP does). This host
> controller is represented by a V4L2 subdevice as well, and the
> connection to the (same) MIPI PHY is established similarly. The
> difference may be that this controller has its registers within the ISP
> register block and is thus tightly coupled to the ISP (and the rkisp1
> driver).
> 
> Now I guess that in split mode the ISP MIPI path works completely
> independent and could be represented either by a separate media graph or
> by another disjoint subgraph in the same media graph.
> In split mode we need to define which host controller should handle
> which set of lanes (there are two sets, clock 1 + data 1 + data 2 as
> well as clock 2 + data 3 + data 4). Right now I am not sure how this
> should be accomplished, but then the split mode can also wait a bit.
> 
> However, as you pointed out in our IRC discussion, it is mentioned in
> TRM Chapter 12 that the RK3568 ISP can also process the DVP input. This
> still needs some verification, the rest of the paragraph is pure
> speculation. But maybe on simply needs to set the correct bits in
> CTRL_0000_VI_DPCL. Now this would mean that there should be one large
> media graph the includes the RK3568 VICAP as well as the RK3568 ISP,
> i.e., something along the lines
> 
>   CC0 (V4L2 subdev) --> RK3568 ISP MIPI CSI-2 HOST (V4L2 subdev)
>                             |
>                             v
>                      RK3568 ISP MUX (V4L2 subdev) --> RK3568 ISP...
>                             ^
>                             |
>   CC1 (V4L2 subdev) --> MAGIC V4L2 subdev? -->
>   RK3568 VICAP DVP (V4L2 device)
> 
>   CC2 (V4L2 subdev) --> RK3568 MIPI CSI-2 HOST (V4L2 subdev) -->
>   RK3568 VICAP MIPI (V4L2 device)
> 
> where the MAGIC V4L2 subdev should be a fan out of some sort?!
> 
> Now to answer your question: I guess it is going to be one media graph
> for RK3568 VICAP and RK3588 ISP. This shall be in perfect alignment with
> the RK3588, where VICAP and ISPs are clearly connected.

>From what I can gather online:

"For RV1126/RV1109 and RK356X platforms, VICAP and ISP are two independent image processing IPs,
If the image collected by VICAP is to be processed by ISP , it is necessary to generate v4l2
corresponding to the interface of VICAP at the driver level sub The device is linked to the node
corresponding to the ISP to provide parameters for the ISP driver to use."

"For the RK356X chip, VICAP has only a single core and has both dvp/mipi interfaces. The dvp interface
corresponds to a rkvicap_dvp node, and the mipi interface corresponds to a rkvicap_mipi_lvds node each
node can be collected independently."

"In order to synchronize the VICAP collected data information to the isp driver, it is necessary to link the
logical sditf node generated by the VICAP driver to the virtual node
device generated by the isp"

In a graph showing how ISP and VICAP are connected a virtual connection
is used (as opposed to a Entitative connection, e.g., between the mipi
sensor and the csi2 dphy)

So from my understanding, VICAP/ISP2.1 of rk3568 seem to be different from the rk3588,
where the VICAP is clearly connected to the two ISP3.0

--
Kind Regards
Mehdi Djait

Powered by blists - more mailing lists