linux-kernel - Re: [PATCH 1/2] staging: media: atomisp: Remove typedefs for basic types in vmem.c

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <aLW-45MfPpLSNi0n@smile.fi.intel.com>
Date: Mon, 1 Sep 2025 18:42:27 +0300
From: Andy Shevchenko <andriy.shevchenko@...el.com>
To: Adrian Barnaś <abarnas@...gle.com>
Cc: Hans de Goede <hansg@...nel.org>,
	Mauro Carvalho Chehab <mchehab@...nel.org>,
	Sakari Ailus <sakari.ailus@...ux.intel.com>,
	Andy Shevchenko <andy@...nel.org>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Dan Carpenter <dan.carpenter@...aro.org>,
	linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-staging@...ts.linux.dev
Subject: Re: [PATCH 1/2] staging: media: atomisp: Remove typedefs for basic
 types in vmem.c

On Mon, Sep 01, 2025 at 03:27:19PM +0200, Adrian Barnaś wrote:
> On Mon, Sep 1, 2025 at 2:35 PM Andy Shevchenko
> <andriy.shevchenko@...el.com> wrote:
> > On Mon, Sep 01, 2025 at 09:10:49AM +0000, Adrian Barnaś wrote:

...

> > > -static inline hive_uedge
> > > -subword(hive_uedge w, unsigned int start, unsigned int end)
> > > +static inline unsigned long long
> > > +subword(unsigned long long w, unsigned int start, unsigned int end)
> > >  {
> > >       return (w & (((1ULL << (end - 1)) - 1) << 1 | 1)) >> start;
> > >  }
> > >
> > >  /* inverse subword bits move like this: MSB[xxxx____xxxx]LSB -> MSB[xxxx0000xxxx]LSB */
> > > -static inline hive_uedge
> > > -inv_subword(hive_uedge w, unsigned int start, unsigned int end)
> > > +static inline unsigned long long
> > > +inv_subword(unsigned long long w, unsigned int start, unsigned int end)
> > >  {
> > >       return w & (~(((1ULL << (end - 1)) - 1) << 1 | 1) | ((1ULL << start) - 1));
> > >  }
> >
> > Also consider to simplify the above (in a separate change).
> >
> > static inline unsigned long long
> > subword(unsigned long long w, unsigned int start, unsigned int end)
> > {
> >         return (w & GENMASK_ULL(end, 0)) >> start;
> > }
> >
> > static inline unsigned long long
> > inv_subword(unsigned long long w, unsigned int start, unsigned int end)
> > {
> >         return w & (~GENMASK_ULL(end, 0) | GENMASK_ULL(start, 0));
> > }
> >
> > ...if I'm not mistaken, so double check all these.
> >
> > At least in my case the end == 64 is not allowed while it seems the original
> > code allows it to be equal to the end == 63 case. Needs testing anyway...
> 
> Those functions works odd:
> when (end = 8, start = 0) it affects bits 0...7

Yes, that's what I meant. But it does the same for 7, 0.
That's why every caller needs to be carefully checked for these corner cases.

64 is special because it will give complete garbage in my case.
I suspect they never extracting anything on the non-aligned byte borders.

> This should make the same results, will check twice if i not missed
> anything and post v2:
> 
> static inline unsigned long long _subword(unsigned long long w,
> unsigned int start,
> unsigned int end)
> {
> return (w & GENMASK_ULL(end-1, 0)) >> start;
> }
> 
> static inline unsigned long long _inv_subword(unsigned long long w,
> unsigned int start,
> unsigned int end)
> {
> return w & (~GENMASK_ULL(end-1, start));
> }

Maybe, but again needs to be carefully checked and tested.

You can propose a separate patch as RFC.

-- 
With Best Regards,
Andy Shevchenko