linux-kernel - Re: [PATCH v2 1/1] lib: Optimise hex_dump_to

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z9HhLr8zD5M1tdGw@smile.fi.intel.com>
Date: Wed, 12 Mar 2025 21:31:58 +0200
From: Andy Shevchenko <andriy.shevchenko@...ux.intel.com>
To: David Laight <david.laight.linux@...il.com>
Cc: linux-kernel@...r.kernel.org, Andrew Morton <akpm@...ux-foundation.org>,
	Arnd Bergmann <arnd@...db.de>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Christophe Leroy <christophe.leroy@....fr>,
	Rasmus Villemoes <linux@...musvillemoes.dk>, nnac123@...ux.ibm.com,
	horms@...nel.org
Subject: Re: [PATCH v2 1/1] lib: Optimise hex_dump_to_buffer()

On Wed, Mar 12, 2025 at 07:18:16PM +0000, David Laight wrote:
> On Mon, 10 Mar 2025 11:05:13 +0200
> Andy Shevchenko <andriy.shevchenko@...ux.intel.com> wrote:
> > On Sat, Mar 08, 2025 at 09:34:21AM +0000, David Laight wrote:

...

> > > -extern int hex_dump_to_buffer(const void *buf, size_t len, int rowsize,
> > > -			      int groupsize, char *linebuf, size_t linebuflen,
> > > -			      bool ascii);
> > > +extern size_t hex_dump_to_buffer(const void *buf, size_t len, size_t rowsize,  
> > > +				 size_t groupsize, char *linebuf,
> > > +				 size_t linebuflen, bool ascii);  
> > 
> > int - > size_t in the returned value is incorrect change.
> > This is explained in the comments to the test cases patch series.
> 
> I don't see you mentioning why.
> The return value is 'the number of bytes that would be output if the buffer
> were large enough' - it is never negative.

True...

> Although given 'a large enough buffer' length is trivially calculable
> it would have been safer to return the actual number of bytes added
> (excluding the '\0').

...but the functions keep the snprintf() semantics, which returns an int.
This makes it more-or-less 1:1 snprintf() substitute in cases where it can
be done in general.

> There were no tests for 'len == 0 && linebuflen == 0', with !ascii the
> existing hex_dump_to_buffer() even manages to return -1.
> (and the function than generates the 'test compare data' is also broken.)

Then you can start with fixes of those?

> Note that libc snprintf() has the same return type as fprintf() which can
> be -1, but any code the looks at is probably broken!
> 
> So an unsigned return type it better.

Maybe, but this will deviate from the prototype and use cases.

-- 
With Best Regards,
Andy Shevchenko