linux-kernel - Re: Uninline kcalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120215201757.GA6934@elliptictech.com>
Date:	Wed, 15 Feb 2012 15:17:57 -0500
From:	Nick Bowler <nbowler@...iptictech.com>
To:	Christoph Lameter <cl@...ux.com>
Cc:	Xi Wang <xi.wang@...il.com>,
	Dan Carpenter <dan.carpenter@...cle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jesper Juhl <jj@...osbits.net>, Jens Axboe <axboe@...nel.dk>,
	Pekka Enberg <penberg@...nel.org>,
	linux-kernel@...r.kernel.org, Matt Mackall <mpm@...enic.com>,
	David Rientjes <rientjes@...gle.com>
Subject: Re: Uninline kcalloc

On 2012-02-14 15:24 -0600, Christoph Lameter wrote:
> On Tue, 14 Feb 2012, Nick Bowler wrote:
> 
> > On 2012-02-14 13:37 -0600, Christoph Lameter wrote:
> > > This patch still preserves kcalloc. But note that if kcalloc returns NULL
> > > then multiple conditions may have caused it. One is that the array is
> > > simply too large. The other may be that such an allocation is not possible
> > > due to fragmentation.
> > [...]
> > > +static inline long calculate_array_size(size_t n, size_t size)
> > > +{
> > > +	if (size != 0 && n > ULONG_MAX / size)
> > > +
> > > +		return 0;
> >
> > This isn't right.  The above tests whether or not the result of the
> > multiplication will not be representable in an 'unsigned long'...
> 
> Yes and so does the current kcalloc.

Well, the current kcalloc doesn't assign the result to a signed long.
However, it does assign the result to a size_t, which makes one wonder
why it's not testing against SIZE_MAX.  If size_t has the same range as
unsigned long on all architectures, then this confusion doesn't matter,
but is that actually the case?

> > > +	return n * size;
> >
> > but then the result is assigned to a (signed) long, which may overflow
> > if it's greater than LONG_MAX.
> 
> That can happen?

Yes, because LONG_MAX (the maximum value of your return type) is
strictly less than ULONG_MAX (what you test against).  It's not hard to
pick input numbers that multiply to something between LONG_MAX and
ULONG_MAX, which will cause your function to return a negative value
(standard C leaves the result of such a conversion implementation-
defined, but I'll assume for now that it works this way for everything
that compiles Linux).

Admittedly, your kcalloc change then assigns this negative value to a
size_t, which will result in the correct positive value assuming
SIZE_MAX == ULONG_MAX, but that's gratuitously roundabout.

[...]
> > [...]
> > >  void *kcalloc(size_t n, size_t size, gfp_t flags)
> > >  {
> > > -	if (size != 0 && n > ULONG_MAX / size)
> > > -		return NULL;
> > > -	return __kmalloc(n * size, flags | __GFP_ZERO);
> > > +	size_t s = calculate_array_size(n, size);
> > > +
> > > +	if (s)
> > > +		return kzalloc(s, flags);
> > > +
> > > +	return NULL;
> > >  }
> >
> > This hunk changes the behaviour of kcalloc if either of the two size parameters
> > is 0.
> 
> You want ZERO_PTR returns?
>
> NULL is one permissible return value of calloc() if size == 0. So we are
> now deviating from user space conventions.

Sort of.  While standard C leaves it implementation-defined whether
successful zero-sized allocations are possible, all sane implementations
let them succeed.  Hence, portable C apps need to handle 0 as a special
case, because there are insane implementations out there.  There's no
reason for the kernel to be one of them.

Regardless, this was still a (presumably unintentional) change from
the previous behaviour.

Cheers,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/