linux-kernel - Re: PATCH: avoid possible integer overflow with cmp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAE9FiQXe2YMN6bL8jdk7fB1ksKCSEFCbL3PF9386vG+NET_0sQ@mail.gmail.com>
Date:	Tue, 25 Nov 2014 13:18:00 -0800
From:	Yinghai Lu <yinghai@...nel.org>
To:	Louis Langholtz <lou_langholtz@...com>
Cc:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	"H. Peter Anvin" <hpa@...ux.intel.com>
Subject: Re: PATCH: avoid possible integer overflow with cmp_range() in kernel/range.c

tile should be:

[PATCH] x86: ....

On Tue, Nov 25, 2014 at 11:14 AM, Louis Langholtz <lou_langholtz@...com> wrote:
> The cmp_range function (in kernel/range.c) is returning the difference between two s64 values (actually coming from u64 typed variables) in an int which can overflow (depending on the size of int). This function is used as a compare function for linux's sort function (in lib/sort.c). Linux's sort function however only cares if the compare function returns a value less than, equal to, or greater than zero.
>
> As sort doesn't need the actual difference, this overflow potential is avoided with the following patch (against linux kernel 3.18 code from Linus's git repo and commit 0541881502a1276149889fe468662ff6a8fc8f6d):
>
> commit 641362d32fef0cfd7b12e1821c1139d75dd23330
> Author: Lou Langholtz <lou_langholtz@...com>
> Date:   Mon Nov 24 09:31:01 2014 -0700

do not need 6 lines.

>
>     Avoid overflow possibility

You need
Signed-off-by: ....

Please check Documentation/SubmittingPatches for more info.

>
> diff --git a/kernel/range.c b/kernel/range.c
> index 322ea8e..86337e2 100644
> --- a/kernel/range.c
> +++ b/kernel/range.c
> @@ -113,12 +113,17 @@ static int cmp_range(const void *x1, const void *x2)
>  {
>         const struct range *r1 = x1;
>         const struct range *r2 = x2;
> -       s64 start1, start2;
> +       u64 start1, start2;
>
>         start1 = r1->start;
>         start2 = r2->start;
>
> -       return start1 - start2;
> +       /* avoid any overflow possibilities and don't just return start1 - start2 */
> +       if (start1 > start2)
> +               return 1;
> +       if (start2 > start1)
> +               return -1;
> +       return 0;
>  }
>
>  int clean_sort_range(struct range *range, int az)
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/