linux-kernel - PATCH: avoid possible integer overflow with cmp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 25 Nov 2014 12:14:15 -0700
From:	Louis Langholtz <lou_langholtz@...com>
To:	linux-kernel@...r.kernel.org
Cc:	yinghai@...nel.org, hpa@...ux.intel.com
Subject: PATCH: avoid possible integer overflow with cmp_range() in
 kernel/range.c

The cmp_range function (in kernel/range.c) is returning the difference between two s64 values (actually coming from u64 typed variables) in an int which can overflow (depending on the size of int). This function is used as a compare function for linux's sort function (in lib/sort.c). Linux's sort function however only cares if the compare function returns a value less than, equal to, or greater than zero.

As sort doesn't need the actual difference, this overflow potential is avoided with the following patch (against linux kernel 3.18 code from Linus's git repo and commit 0541881502a1276149889fe468662ff6a8fc8f6d):

commit 641362d32fef0cfd7b12e1821c1139d75dd23330
Author: Lou Langholtz <lou_langholtz@...com>
Date:   Mon Nov 24 09:31:01 2014 -0700

    Avoid overflow possibility

diff --git a/kernel/range.c b/kernel/range.c
index 322ea8e..86337e2 100644
--- a/kernel/range.c
+++ b/kernel/range.c
@@ -113,12 +113,17 @@ static int cmp_range(const void *x1, const void *x2)
 {
        const struct range *r1 = x1;
        const struct range *r2 = x2;
-       s64 start1, start2;
+       u64 start1, start2;

        start1 = r1->start;
        start2 = r2->start;

-       return start1 - start2;
+       /* avoid any overflow possibilities and don't just return start1 - start2 */
+       if (start1 > start2)
+               return 1;
+       if (start2 > start1)
+               return -1;
+       return 0;
 }

 int clean_sort_range(struct range *range, int az)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/