linux-kernel - Re: [PATCH v3 15/22] staging: media: tegra-video: tegra20: simplify format align calculations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <DDBGU9ELXIAW.1RLHSNOPVR9B3@bootlin.com>
Date: Mon, 06 Oct 2025 20:54:38 +0200
From: "Luca Ceresoli" <luca.ceresoli@...tlin.com>
To: "Svyatoslav Ryhel" <clamor95@...il.com>, "Mikko Perttunen"
 <mperttunen@...dia.com>
Cc: "David Airlie" <airlied@...il.com>, "Simona Vetter" <simona@...ll.ch>,
 "Maarten Lankhorst" <maarten.lankhorst@...ux.intel.com>, "Maxime Ripard"
 <mripard@...nel.org>, "Thomas Zimmermann" <tzimmermann@...e.de>, "Rob
 Herring" <robh@...nel.org>, "Krzysztof Kozlowski" <krzk+dt@...nel.org>,
 "Conor Dooley" <conor+dt@...nel.org>, "Thierry Reding"
 <thierry.reding@...il.com>, "Jonathan Hunter" <jonathanh@...dia.com>,
 "Sowjanya Komatineni" <skomatineni@...dia.com>, "Prashant Gaikwad"
 <pgaikwad@...dia.com>, "Michael Turquette" <mturquette@...libre.com>,
 "Stephen Boyd" <sboyd@...nel.org>, "Linus Walleij"
 <linus.walleij@...aro.org>, "Mauro Carvalho Chehab" <mchehab@...nel.org>,
 "Greg Kroah-Hartman" <gregkh@...uxfoundation.org>,
 Jonas Schwöbel <jonasschwoebel@...oo.de>, "Dmitry
 Osipenko" <digetx@...il.com>, "Charan Pedumuru"
 <charan.pedumuru@...il.com>, "Diogo Ivo" <diogo.ivo@...nico.ulisboa.pt>,
 "Aaron Kling" <webgeek1234@...il.com>, "Arnd Bergmann" <arnd@...db.de>,
 <dri-devel@...ts.freedesktop.org>, <devicetree@...r.kernel.org>,
 <linux-tegra@...r.kernel.org>, <linux-kernel@...r.kernel.org>,
 <linux-media@...r.kernel.org>, <linux-clk@...r.kernel.org>,
 <linux-gpio@...r.kernel.org>, <linux-staging@...ts.linux.dev>
Subject: Re: [PATCH v3 15/22] staging: media: tegra-video: tegra20: simplify
 format align calculations

Hello Svyatoslav,

On Thu Oct 2, 2025 at 8:20 AM CEST, Svyatoslav Ryhel wrote:
>> > > > 12 represents amount of bits used per pixel, 8 for Y plane, 2 for U
>> > > > plane and 2 for V plane, total is 12. "but explainable with a comment
>> > > > and improve-able later" why then we cannot use 12 with a comment? this
>> > > > is all arbitrary. Downstream is not wrong from this perspective, you
>> > > > don't take into account that YUV420 is planar and it uses 3 planes a
>> > > > whole Y plane and 1/4 of U and V which in total results in wigth + 2 *
>> > > > 1/4 width which is width * 3/2
>> > >
>> > > Yes -- but AIUI, the only thing the bpp value is used for the bytesperline calculation. When we add the special case for planar formats, which doesn't use the bpp value, then the value 12 is never used anywhere. We should at least have a comment saying it is unused. (At that point, we could just hardcode the bpp values in the fmt_align function -- but I don't mind either way.)
>> > >
>> > https://ffmpeg.org/pipermail/ffmpeg-user/2023-June/056488.html
>>
>> I understand very well that for YUV420, each pixel has 12 bits of color information. But how many bits of color information each pixel has is not useful in the context of this driver. The number of bytes per line is not related to how many bits of color information each pixel has for planar formats.
>
> No, it has direct impact. This is how buffer size / image size is
> calculated since we place each plane consecutive. And bytes per line
> is used specifically in image size calculation. This is common part
> with non-planar formats. Then since Tegra provides a dedicated
> channels/buffers for each plane, configuration of planar format
> includes an additional step with calculation for each plane.

Sorry, I haven't followed the discussion in detail, but I tested you series
on Tegra20 VIP and capture does not work, with a SIGSEGV in
gstreamer. Bisecting pointed to this as the first commit where the issue
happens.

I compared the input and output values of tegra20_fmt_align() at this
commit and at the previous one, and this is the result:

                       before this patch     with this patch
  At function entry:
  bpp                        1                     12
  pix->width                 640                   640
  pix->height                480                   480
  		           
  On return:        
  pix->bytesperline          640                   960
  pix->sizeimage             460800                460800

I hope these info will help.

Best regards,
Luca

--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com