Message-ID: <20260122095308.GF209830@killaraus>
Date: Thu, 22 Jan 2026 11:53:08 +0200
From: Laurent Pinchart <laurent.pinchart@...asonboard.com>
To: Jai Luthra <jai.luthra@...asonboard.com>
Cc: Tomi Valkeinen <tomi.valkeinen@...asonboard.com>,
	Sakari Ailus <sakari.ailus@...ux.intel.com>,
	y-abhilashchandra@...com, devarsht@...com, s-jain1@...com,
	vigneshr@...com, mchehab@...nel.org, robh@...nel.org,
	krzk+dt@...nel.org, p.zabel@...gutronix.de, conor+dt@...nel.org,
	hverkuil-cisco@...all.nl, changhuang.liang@...rfivetech.com,
	jack.zhu@...rfivetech.com, sjoerd@...labora.com,
	dan.carpenter@...aro.org, hverkuil+cisco@...nel.org,
	linux-kernel@...r.kernel.org, linux-media@...r.kernel.org,
	devicetree@...r.kernel.org, jai.luthra@...ux.dev,
	mripard@...nel.org, Rishikesh Donadkar <r-donadkar@...com>
Subject: Re: [PATCH v9 06/19] media: ti: j721e-csi2rx: add a subdev for the
 core device

On Thu, Jan 22, 2026 at 12:23:50PM +0530, Jai Luthra wrote:
> Quoting Laurent Pinchart (2026-01-21 16:22:32)
> > On Wed, Jan 21, 2026 at 09:38:29AM +0200, Tomi Valkeinen wrote:
> > > On 21/01/2026 01:25, Laurent Pinchart wrote:
> > > > On Thu, Jan 15, 2026 at 02:56:21PM +0200, Tomi Valkeinen wrote:
> > > >> On 15/01/2026 08:36, Jai Luthra wrote:
> > > >>> Quoting Tomi Valkeinen (2026-01-14 20:51:49)
> > > >>>> On 30/12/2025 10:32, Rishikesh Donadkar wrote:
> > > >>>>> From: Jai Luthra <j-luthra@...com>
> > > >>>>>
> > > >>>>> With single stream capture, it was simpler to use the video device as
> > > >>>>> the media entity representing the main TI CSI2RX device. Now with multi
> > > >>>>> stream capture coming into the picture, the model has shifted to each
> > > >>>>> video device having a link to the main device's subdev. The routing
> > > >>>>> would then be set on this subdev.
> > > >>>>>
> > > >>>>> Add this subdev, link each context to this subdev's entity and link the
> > > >>>>> subdev's entity to the source. Also add an array of media pads. It will
> > > >>>>> have one sink pad and source pads equal to the number of contexts.
> > > >>>>>
> > > >>>>> Support the new enable_stream()/disable_stream() APIs in the subdev
> > > >>>>> instead of s_stream() hook.
> > > >>>>>
> > > >>>>> Reviewed-by: Yemike Abhilash Chandra <y-abhilashchandra@...com>
> > > >>>>> Co-developed-by: Pratyush Yadav <p.yadav@...com>
> > > >>>>> Signed-off-by: Pratyush Yadav <p.yadav@...com>
> > > >>>>> Signed-off-by: Jai Luthra <j-luthra@...com>
> > > >>>>> Signed-off-by: Rishikesh Donadkar <r-donadkar@...com>
> > > >>>>> ---
> > > >>>
> > > >>> [...]
> > > >>>
> > > >>>>> @@ -981,48 +1138,52 @@ static int ti_csi2rx_link_validate(struct media_link *link)
> > > >>>>>       struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
> > > >>>>>       struct ti_csi2rx_dev *csi = ctx->csi;
> > > >>>>>       struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
> > > >>>>> -     struct v4l2_subdev_format source_fmt = {
> > > >>>>> -             .which  = V4L2_SUBDEV_FORMAT_ACTIVE,
> > > >>>>> -             .pad    = link->source->index,
> > > >>>>> -     };
> > > >>>>> +     struct v4l2_mbus_framefmt *format;
> > > >>>>> +     struct v4l2_subdev_state *state;
> > > >>>>>       const struct ti_csi2rx_fmt *ti_fmt;
> > > >>>>> -     int ret;
> > > >>>>>  
> > > >>>>> -     ret = v4l2_subdev_call_state_active(csi->source, pad,
> > > >>>>> -                                         get_fmt, &source_fmt);
> > > >>>>> -     if (ret)
> > > >>>>> -             return ret;
> > > >>>>> +     state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
> > > >>>>> +     format = v4l2_subdev_state_get_format(state, link->source->index, 0);
> > > >>>>> +     v4l2_subdev_unlock_state(state);
> > > >>>>>  
> > > >>>>> -     if (source_fmt.format.width != csi_fmt->width) {
> > > >>>>> +     if (!format) {
> > > >>>>> +             dev_dbg(csi->dev,
> > > >>>>> +                     "Skipping validation as no format present on \"%s\":%u:0\n",
> > > >>>>> +                     link->source->entity->name, link->source->index);
> > > >>>>> +             return 0;
> > > >>>>
> > > >>>> Isn't this an error?
> > > >>>
> > > >>> Well, the j7 shim subdev introduced here has immutable and active links to
> > > >>> all the video nodes, for each DMA channel (taken from DT), many of which
> > > >>> may be unused for certain setups, and thus there might not be any valid
> > > >>> format on the subdev source pad corresponding to an unused video node.
> > > >>>
> > > >>> Jacopo had a similar comment on v2, see this discussion (grep for Mali):
> > > >>> https://lore.kernel.org/linux-media/4mnlnsj4co3agvln4qsasmgvgwiyoo7yu2h5wyh4rmzzafhm5u@avhnbw7iknms/
> > > >>>
> > > >>> I know other drivers use a different approach with mutable links, so it
> > > >>> would be good if you/Laurent/Sakari can give your opinions on if only one
> > > >>> of these two approaches should be taken for multi-stream pipelines.
> > > >>
> > > >> I see.
> > > >>
> > > >> Well, I don't have a definite answer. With some thinking both options
> > > >> make certain sense. It makes sense to keep the links immutable and
> > > >> always enabled, as there's no configuration that can be done. On the
> > > >> other hand, it makes sense to require the unused links to be disabled,
> > > >> as, well, they are not used.
> > > > 
> > > > I'm not familiar with the implications this would have on this driver,
> > > > but generally speaking, if a stream is added to the media pipeline by
> > > > the pipeline build algorithm, then it is expected that applications
> > > > would have configured it correctly. Streams that are not used are
> > > > expected to be disabled if they would otherwise be added to the
> > > > pipeline.
> > > 
> > > I think the thing here is that the driver creates immutable
> > > always-enabled media links between the videodevs and the first subdev.
> > > Then, say, if only one stream is being used, only one of those links is
> > > actually used, and for every other link the above check fails as there's
> > > no stream, so no format.
> > > 
> > > In TI CAL driver the links were mutable, and unused links had to be
> > > disabled. There it made sense as the links had to be configurable (there
> > > were two PHYs). Here, there's no configuration needed, so immutable
> > > links make sense, but then they're enabled even when actually not used.
> > 
> > If the routing table in the subdev does not contain any route that goes
> > towards a video node, then that video node should not be added to the
> > pipeline by the validation code, and no validation will be attempted. At
> > least that's the theory.
> 
> Okay that sounds reasonable. I can take a look into the media pipeline
> validation code next week. @Rishikesh, given you already have a working
> setup, feel free to test if the link_validate callback is triggered on
> video nodes that don't have any streams/routes pointing to them.
> 
> > I see that this driver implements .link_validate() as a
> > media_entity_operations, not a subdev operation. I wonder if that could
> > explain the issue.
> 
> Well earlier I was partially confused, now I'm fully confused :-)
> 
> How is v4l2_subdev_pad_ops.link_validate different from
> media_entity_operations.link_validate?

media_entity_operations.link_validate() is the entry point, called by
the media pipeline validation code that is agnostic to entity types. It
is called on the sink entity of each link.

When the sink is a subdev, the common practice is to implement
media_entity_operations.link_validate() using
v4l2_subdev_link_validate(), which iterates over streams and validate
them individually with v4l2_subdev_pad_ops.link_validate(). That
operation can be left out if the default implementation
v4l2_subdev_link_validate_default() is enough, or a custom
.link_validate() operation can be provided that calls
v4l2_subdev_link_validate_default() and performs additional checks (or
implements all checks manually without calling
v4l2_subdev_link_validate_default() for very uncommon cases).

In this case, though, the sink is a video device, so
v4l2_subdev_link_validate() can't be used. I overlooked that in my
previous reply, sorry about it. As a link to a video node can only carry
a single stream, it seems that the issue here is caused by the link
being included in the media pipeline in the first place.
drivers/media/mc/mc-entity.c contains detailed debug messages that
explain how a pipeline is constructed, I would start by enabling them
and investigating what happens.

> I see mc-core.rst and v4l2-subdev.rst both talk about their own variant,
> without making it clear which should be used for a subdev.
> 
> Anyway, I'll try to dig through the framework code to understand what's
> going wrong.

-- 
Regards,

Laurent Pinchart

Powered by blists - more mailing lists