linux-kernel - Re: [PATCH v3 00/24] i.MX Media Driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3a673cba-bbf6-5611-5857-4605797bf049@gmail.com>
Date:   Tue, 31 Jan 2017 15:43:22 -0800
From:   Steve Longerbeam <slongerbeam@...il.com>
To:     Russell King - ARM Linux <linux@...linux.org.uk>
Cc:     mark.rutland@....com, andrew-ct.chen@...iatek.com,
        minghsiu.tsai@...iatek.com, nick@...anahar.org,
        songjun.wu@...rochip.com, hverkuil@...all.nl,
        Steve Longerbeam <steve_longerbeam@...tor.com>,
        robert.jarzmik@...e.fr, devel@...verdev.osuosl.org,
        markus.heiser@...marIT.de,
        laurent.pinchart+renesas@...asonboard.com, geert@...ux-m68k.org,
        linux-media@...r.kernel.org, devicetree@...r.kernel.org,
        kernel@...gutronix.de, arnd@...db.de, mchehab@...nel.org,
        bparrot@...com, robh+dt@...nel.org, horms+renesas@...ge.net.au,
        tiffany.lin@...iatek.com, linux-arm-kernel@...ts.infradead.org,
        niklas.soderlund+renesas@...natech.se, gregkh@...uxfoundation.org,
        linux-kernel@...r.kernel.org, jean-christophe.trotin@...com,
        p.zabel@...gutronix.de, fabio.estevam@....com, shawnguo@...nel.org,
        sudipm.mukherjee@...il.com
Subject: Re: [PATCH v3 00/24] i.MX Media Driver



On 01/31/2017 03:00 AM, Russell King - ARM Linux wrote:
> On Mon, Jan 30, 2017 at 05:22:01PM -0800, Steve Longerbeam wrote:
>> I'm also having trouble finding a datasheet for it, but from what
>> I've read, it has a MIPI CSI-2 interface. It should work fine as long
>> as it presents a single source pad, registers asynchronously, and
>> sets its entity function to MEDIA_ENT_F_CAM_SENSOR.
> Yes, it is MIPI CSI-2, and yes it has a single source pad, registers
> asynchronously, but that's about as far as it goes.
>
> The structure is a camera sensor followed by some processing.  So just
> like the smiapp code, I've ended up with multiple subdevs describing
> each stage of the sensors pipeline.
>
> Just like smiapp, the camera sensor block (which is the very far end
> of the pipeline) is marked with MEDIA_ENT_F_CAM_SENSOR.  However, in
> front of that is the binner, which just like smiapp gets a separate
> entity.  It's this entity which is connected to the mipi-csi2 subdev.

wow, ok got it.

So the sensor pipeline and binner, and the OF graph connecting
them, are described in the device tree I presume.

The OF graph AFAIK, has no information about which ports are sinks
and which are sources, so of_parse_subdev() tries to determine that
based on the compatible string of the device node. So ATM
of_parse_subdev() assumes there is nothing but the imx6-mipi-csi2,
video-multiplexer, and camera sensors upstream from the CSI ports
in the OF graph.

I realize that's not a robust solution, and is the reason for the
"no sensor attached" below.

Is there any way to determine from the OF graph the data-direction
of a port (whether it is a sink or a source)? If so it will make
of_parse_subdev() much more robust.

Steve

>
> Unlike smiapp, which does not set an entity function, I set my binner
> entity as MEDIA_ENT_F_PROC_VIDEO_SCALER on the basis that that is
> what V4L2 documentation recommend:
>
>      -  ..  row 27
>
>         ..  _MEDIA-ENT-F-PROC-VIDEO-SCALER:
>
>         -  ``MEDIA_ENT_F_PROC_VIDEO_SCALER``
>
>         -  Video scaler. An entity capable of video scaling must have
>            at least one sink pad and one source pad, and scale the
>            video frame(s) received on its sink pad(s) to a different
>            resolution output on its source pad(s). The range of
>            supported scaling ratios is entity-specific and can differ
>            between the horizontal and vertical directions (in particular
>            scaling can be supported in one direction only). Binning and
>            skipping are considered as scaling.
>
> This causes attempts to configure the ipu1_csi0 interface to fail:
>
> media-ctl -v -d /dev/media1 --set-v4l2 '"ipu1_csi0":1[fmt:SGBRG8/512x512@...0]'
> Opening media device /dev/media1
> Enumerating entities
> Found 29 entities
> Enumerating pads and links
> Setting up format SGBRG8 512x512 on pad ipu1_csi0/1
> Unable to set format: No such device (-19)
> Unable to setup formats: No such device (19)
>
> and in the kernel log:
>
> ipu1_csi0: no sensor attached
>
> And yes, I already know that my next problem is going to be that the bayer
> formats are not supported in your driver (just like Philipp's driver) but
> adding them should not be difficult... but only once this issue is resolved.
>