linux-kernel - [PATCH v1 1/2] tools/nolibc/stdlib: Support overflow checking for older compiler versions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 20 May 2022 00:21:15 +0700
From:   Ammar Faizi <ammarfaizi2@...weeb.org>
To:     "Paul E. McKenney" <paulmck@...nel.org>, Willy Tarreau <w@....eu>
Cc:     Ammar Faizi <ammarfaizi2@...weeb.org>,
        Alviro Iskandar Setiawan <alviro.iskandar@...weeb.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        GNU/Weeb Mailing List <gwml@...r.gnuweeb.org>,
        Facebook Kernel Team <kernel-team@...com>
Subject: [PATCH v1 1/2] tools/nolibc/stdlib: Support overflow checking for older compiler versions

Previously, we used __builtin_mul_overflow() to check for overflow in
the multiplication operation in the calloc() function. However, older
compiler versions don't support this built-in. This patch changes the
overflow checking mechanism to make it work on any compiler version
by using a division method to check for overflow. No functional change
intended. While in there, remove the unused variable `void *orig`.

Link: https://lore.kernel.org/lkml/20220330024114.GA18892@1wt.eu
Suggested-by: Willy Tarreau <w@....eu>
Cc: Alviro Iskandar Setiawan <alviro.iskandar@...weeb.org>
Signed-off-by: Ammar Faizi <ammarfaizi2@...weeb.org>
---
 tools/include/nolibc/stdlib.h | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tools/include/nolibc/stdlib.h b/tools/include/nolibc/stdlib.h
index 8fd32eaf8037..92378c4b9660 100644
--- a/tools/include/nolibc/stdlib.h
+++ b/tools/include/nolibc/stdlib.h
@@ -128,10 +128,9 @@ void *malloc(size_t len)
 static __attribute__((unused))
 void *calloc(size_t size, size_t nmemb)
 {
-	void *orig;
-	size_t res = 0;
+	size_t x = size * nmemb;
 
-	if (__builtin_expect(__builtin_mul_overflow(nmemb, size, &res), 0)) {
+	if (__builtin_expect(size && ((x / size) != nmemb), 0)) {
 		SET_ERRNO(ENOMEM);
 		return NULL;
 	}
@@ -140,7 +139,7 @@ void *calloc(size_t size, size_t nmemb)
 	 * No need to zero the heap, the MAP_ANONYMOUS in malloc()
 	 * already does it.
 	 */
-	return malloc(res);
+	return malloc(x);
 }
 
 static __attribute__((unused))
-- 
Ammar Faizi