linux-kernel - Re: Uninline kcalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 14 Feb 2012 15:50:32 -0500
From:	Nick Bowler <nbowler@...iptictech.com>
To:	Christoph Lameter <cl@...ux.com>
Cc:	Xi Wang <xi.wang@...il.com>,
	Dan Carpenter <dan.carpenter@...cle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jesper Juhl <jj@...osbits.net>, Jens Axboe <axboe@...nel.dk>,
	Pekka Enberg <penberg@...nel.org>,
	linux-kernel@...r.kernel.org, Matt Mackall <mpm@...enic.com>,
	David Rientjes <rientjes@...gle.com>
Subject: Re: Uninline kcalloc

On 2012-02-14 13:37 -0600, Christoph Lameter wrote:
> This patch still preserves kcalloc. But note that if kcalloc returns NULL
> then multiple conditions may have caused it. One is that the array is
> simply too large. The other may be that such an allocation is not possible
> due to fragmentation.
[...]
> +static inline long calculate_array_size(size_t n, size_t size)
> +{
> +	if (size != 0 && n > ULONG_MAX / size)
> +
> +		return 0;

This isn't right.  The above tests whether or not the result of the
multiplication will not be representable in an 'unsigned long'...

> +
> +	return n * size;

but then the result is assigned to a (signed) long, which may overflow 
if it's greater than LONG_MAX.

> +}
[...]
>  void *kcalloc(size_t n, size_t size, gfp_t flags)
>  {
> -	if (size != 0 && n > ULONG_MAX / size)
> -		return NULL;
> -	return __kmalloc(n * size, flags | __GFP_ZERO);
> +	size_t s = calculate_array_size(n, size);
> +
> +	if (s)
> +		return kzalloc(s, flags);
> +
> +	return NULL;
>  }

This hunk changes the behaviour of kcalloc if either of the two size parameters
is 0.

Cheers,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/