linux-kernel - kcalloc/kmalloc_array could BUILD_BUG_ON for too-big constant arguments (was Re: [PATCH] [RESEND] aic7xxx: replace kmalloc/memset by kzalloc)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20150327194824.GC88629@unpythonic.net>
Date:	Fri, 27 Mar 2015 14:48:24 -0500
From:	Jeff Epler <jepler@...ythonic.net>
To:	Michael Opdenacker <michael.opdenacker@...e-electrons.com>
Cc:	"Elliott, Robert (Server Storage)" <Elliott@...com>,
	Joe Perches <joe@...ches.com>, Hannes Reinecke <hare@...e.de>,
	"JBottomley@...allels.com" <JBottomley@...allels.com>,
	"linux-scsi@...r.kernel.org" <linux-scsi@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: kcalloc/kmalloc_array could BUILD_BUG_ON for too-big constant
 arguments (was Re: [PATCH] [RESEND] aic7xxx: replace kmalloc/memset by
 kzalloc)

The following is a sketch of how a macro kcalloc could BUILD_BUG_ON for
overflows of two compile-time operands, or call "kcalloc_variable" for
nonconstant arguments.  Tested on gcc 4.7.2 only, since it's what I had to
hand.  I didn't do any testing beyond checking that fn2 didn't build, and that
fn1/3 had plausible-looking code on x86_64.

typedef unsigned long size_t;
#define SIZE_MAX (~(size_t)0)
typedef int gfp_t;
extern void *kzalloc(size_t n, gfp_t flags);
extern void *kcalloc_variable(size_t n, size_t size, gfp_t flags);
#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))

#define kcalloc(n, size, flags) \
	__builtin_choose_expr(__builtin_constant_p((n) | (size)), \
		( \
			BUILD_BUG_ON((n) > SIZE_MAX / (size)), \
			kzalloc((n) * (size), (flags)) \
		), kcalloc_variable((n), (size), (flags)))


void fn1() { kcalloc(3, 3, 0); }
//void fn2() { kcalloc(2, ~(size_t)0, 0); }// compile-time BUILD_BUG_ON
void fn3(int i) { kcalloc(2, i, 0); }

Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/