linux-kernel - Re: [PATCH] atm: he: fix sign-extension overflow on large shift

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 16 Jan 2019 11:50:51 +0300
From:   Dan Carpenter <dan.carpenter@...cle.com>
To:     Colin King <colin.king@...onical.com>
Cc:     Chas Williams <3chas3@...il.com>,
        linux-atm-general@...ts.sourceforge.net, netdev@...r.kernel.org,
        kernel-janitors@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] atm: he: fix sign-extension overflow on large shift

On Tue, Jan 15, 2019 at 06:03:38PM +0000, Colin King wrote:
> From: Colin Ian King <colin.king@...onical.com>
> 
> Shifting the 1 by exp by an int can lead to sign-extension overlow when
> exp is 31 since 1 is an signed int and sign-extending this result to an
> unsigned long long will set the upper 32 bits.  Fix this by shifting an
> unsigned long.

You could just cast it to unsigned.

	rate_cps = (unsigned long long) (1U << exp) * (man + 512) >> 9;

I don't think it makes a difference at runtime because we just
pick buf = 4 either way...

regards,
dan carpenter