linux-kernel - Re: Uninline kcalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.1202141513210.29019@router.home>
Date:	Tue, 14 Feb 2012 15:24:58 -0600 (CST)
From:	Christoph Lameter <cl@...ux.com>
To:	Nick Bowler <nbowler@...iptictech.com>
cc:	Xi Wang <xi.wang@...il.com>,
	Dan Carpenter <dan.carpenter@...cle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jesper Juhl <jj@...osbits.net>, Jens Axboe <axboe@...nel.dk>,
	Pekka Enberg <penberg@...nel.org>,
	linux-kernel@...r.kernel.org, Matt Mackall <mpm@...enic.com>,
	David Rientjes <rientjes@...gle.com>
Subject: Re: Uninline kcalloc

On Tue, 14 Feb 2012, Nick Bowler wrote:

> On 2012-02-14 13:37 -0600, Christoph Lameter wrote:
> > This patch still preserves kcalloc. But note that if kcalloc returns NULL
> > then multiple conditions may have caused it. One is that the array is
> > simply too large. The other may be that such an allocation is not possible
> > due to fragmentation.
> [...]
> > +static inline long calculate_array_size(size_t n, size_t size)
> > +{
> > +	if (size != 0 && n > ULONG_MAX / size)
> > +
> > +		return 0;
>
> This isn't right.  The above tests whether or not the result of the
> multiplication will not be representable in an 'unsigned long'...

Yes and so does the current kcalloc.

> > +	return n * size;
>
> but then the result is assigned to a (signed) long, which may overflow
> if it's greater than LONG_MAX.

That can happen? But you are right its better to use the size_t as the
result of the argument. There is a gooh of long / size_t confusion
already. This useless function should not be in the kernel.


> > +}
> [...]
> >  void *kcalloc(size_t n, size_t size, gfp_t flags)
> >  {
> > -	if (size != 0 && n > ULONG_MAX / size)
> > -		return NULL;
> > -	return __kmalloc(n * size, flags | __GFP_ZERO);
> > +	size_t s = calculate_array_size(n, size);
> > +
> > +	if (s)
> > +		return kzalloc(s, flags);
> > +
> > +	return NULL;
> >  }
>
> This hunk changes the behaviour of kcalloc if either of the two size parameters
> is 0.

You want ZERO_PTR returns?

NULL is one permissible return value of calloc() if size == 0. So we are
now deviating from user space conventions.



Subject: Introduce calculate_array_size

calculate_array_size calculates the size of an array while
checking for errors. Returns 0 if the size is too large.

Signed-off-by: Christoph Lameter <cl@...ux.com>


---
 include/linux/slab.h |   15 +++++++++++++++
 mm/util.c            |    7 +++++--
 2 files changed, 20 insertions(+), 2 deletions(-)

Index: linux-2.6/include/linux/slab.h
===================================================================
--- linux-2.6.orig/include/linux/slab.h	2012-02-14 15:16:53.000000000 -0600
+++ linux-2.6/include/linux/slab.h	2012-02-14 15:21:39.000000000 -0600
@@ -242,6 +242,21 @@ size_t ksize(const void *);
  */
 void *kcalloc(size_t n, size_t size, gfp_t flags);

+/*
+ * calculate_array_size - Calculate an array size given the size of a
+ * particular element with checking for overflow.
+ *
+ * Return 0 if there is an overflow.
+ */
+static inline size_t calculate_array_size(size_t n, size_t size)
+{
+	if (n > ULONG_MAX / size)
+
+		return 0;
+
+	return n * size;
+}
+
 #if !defined(CONFIG_NUMA) && !defined(CONFIG_SLOB)
 /**
  * kmalloc_node - allocate memory from a specific node
Index: linux-2.6/mm/util.c
===================================================================
--- linux-2.6.orig/mm/util.c	2012-02-14 15:16:53.000000000 -0600
+++ linux-2.6/mm/util.c	2012-02-14 15:21:05.000000000 -0600
@@ -83,9 +83,12 @@ EXPORT_SYMBOL(kmemdup);
  */
 void *kcalloc(size_t n, size_t size, gfp_t flags)
 {
-	if (size != 0 && n > ULONG_MAX / size)
+	size_t s = calculate_array_size(n, size);
+
+	if (size && !s)
 		return NULL;
-	return __kmalloc(n * size, flags | __GFP_ZERO);
+
+	return kzalloc(s, flags);
 }
 EXPORT_SYMBOL(kcalloc);

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/