linux-kernel - Re: [PATCH v2] media: i2c: ov5640: Implement get_mbus

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAPY8ntCnX9uFbyfKnJCH6+8yLWwX1ZieYqzZq6qs9uvAPh9Eyw@mail.gmail.com>
Date:   Mon, 20 Mar 2023 18:31:04 +0000
From:   Dave Stevenson <dave.stevenson@...pberrypi.com>
To:     Laurent Pinchart <laurent.pinchart@...asonboard.com>
Cc:     Philipp Zabel <p.zabel@...gutronix.de>,
        Sakari Ailus <sakari.ailus@...ux.intel.com>,
        Jacopo Mondi <jacopo.mondi@...asonboard.com>,
        Francesco Dolcini <francesco@...cini.it>,
        Sakari Ailus <sakari.ailus@....fi>,
        Marcel Ziswiler <marcel@...wiler.com>,
        linux-media@...r.kernel.org, kernel@...gutronix.de,
        Francesco Dolcini <francesco.dolcini@...adex.com>,
        Aishwarya Kothari <aishwarya.kothari@...adex.com>,
        Marcel Ziswiler <marcel.ziswiler@...adex.com>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Steve Longerbeam <slongerbeam@...il.com>,
        linux-kernel@...r.kernel.org,
        Marco Felsch <m.felsch@...gutronix.de>
Subject: Re: [PATCH v2] media: i2c: ov5640: Implement get_mbus_config

On Mon, 20 Mar 2023 at 14:00, Laurent Pinchart
<laurent.pinchart@...asonboard.com> wrote:
>
> On Mon, Mar 20, 2023 at 02:32:25PM +0100, Philipp Zabel wrote:
> > On Mo, 2023-03-20 at 12:15 +0200, Sakari Ailus wrote:
> > > On Mon, Mar 20, 2023 at 11:55:14AM +0200, Laurent Pinchart wrote:
> > > > On Mon, Mar 20, 2023 at 11:37:33AM +0200, Sakari Ailus wrote:
> > > > > On Mon, Mar 20, 2023 at 11:26:02AM +0200, Laurent Pinchart wrote:
> > > > > > In a (simplified) nutshell,
> > > > > >
> > > > > > ---------+     +----------+     +---------+     +-----+     +-----+
> > > > > > > Camera | --> | CSI-2 RX | --> | CSI2IPU | --> | Mux | --> | IPU |
> > > > > > > Sensor |     |          |     | Gasket  |     |     |     |     |
> > > > > > ---------+     +----------+     +---------+     +-----+     +-----+
> > > > >
> > > > > Thank you, this is helpful.
> > > > >
> > > > > I suppose the mux here at least won't actively do anything to the data. So
> > > > > presumably its endpoint won't contain the active configuration, but its
> > > > > superset.
> > > > >
> > > > > >
> > > > > > All those blocks, except for the gasket, have a node in DT.
> > > > > >
> > > > > > The IPU driver needs to know the number of CSI-2 data lanes, which is
> > > > > > encoded in the data-lanes DT property present in both the sensor output
> > > > > > endpoint and the CSI-2 RX input endpoint, but not the other endpoints in
> > > > > > the pipeline.
> > > > >
> > > > > This doesn't yet explain why the sensor would need to implement
> > > > > get_mbus_config if its bus configuration remains constant.
> > > >
> > > > If I recall correctly, the IPU driver calls .g_mbus_config() on the
> > > > camera sensor to get the number of lanes, as it can't get it from its
> > > > own endpoint. That's a hack, and as Jacopo proposed, calling
> > > > .g_mbus_config() on the CSI-2 RX would be better, as the CSI-2 RX driver
> > > > can then get the value from its own endpoint, without requiring all
> > > > sensor drivers to implement .g_mbus_config().
> > >
> > > The g_mbus_config op could be implemented by the CSI2IPU and mux, by simply
> > > requesting the information from the upstream sub-device. No hacks would be
> > > needed.
> >
> > I think implementing get_mbus_config on the mux might be enough. The
> > IPU driver already recognizes the CSI-2 RX by its grp_id and could
> > infer that it has to use MIPI CSI-2 mode from that.
> >
> > The video-mux would have to forward get_mbus_config to its active
> > upstream port and if the upstream sensor does not implement it read bus
> > width from the active upstream endpoint.
>
> I'm fine with implementing it in the mux as well, but I think we can
> take a shortcut here and call it on the CSI-2 RX from the IPU driver, as
> the IPU driver knows about the architecture of the whole pipeline.

FWIW I have made use of video-mux and implementing g_mbus_config on it
for D-PHY switch chips[1] where the different input ports may have
different configurations. I'll admit that I've made the easy
assumption that it's CSI-2 D-PHY in and out, when it could quite
legitimately be working with any of the other bus types.

I had been intending to send this[2] upstream when I get a chance, but
am I reading imx6q.dtsi[3] correctly in that the mux is accepting
parallel on some ports and CSI-2 on others? The mux hardware is
therefore far more than just a simple switch between inputs? Although
as this is after the CSI2 rx block, this is effectively parallel data
within the SoC, therefore is the configuration and get_mbus_config
really relevant?
I'd like to understand how this is being used on imx6 before trying to
rework my patch into a generic solution.

Thanks
  Dave

[1] eg Arducam's 2 and 4 port muxes -
https://www.arducam.com/product-category/camera-multiplexers/, which
IIRC use OnSemi's FSA644
https://www.onsemi.com/products/interfaces/analog-switches/fsa644
[2] https://github.com/raspberrypi/linux/commit/bf653318475cf4db0ec59e92139f477f7cc0a544
[3] https://elixir.bootlin.com/linux/latest/source/arch/arm/boot/dts/imx6q.dtsi#L349