linux-kernel - Re: [PATCH] avoid negative shifts in radix-tree.c, take 2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <jeodgq1p9m.fsf@sykes.suse.de>
Date:	Wed, 29 Aug 2007 14:47:17 +0200
From:	Andreas Schwab <schwab@...e.de>
To:	Peter Lund <firefly@...64.dk>
Cc:	Christoph Lameter <clameter@....com>,
	Christoph Hellwig <hch@...radead.org>, trivial@...nel.org,
	linux-kernel@...r.kernel.org, Momchil Velikov <velco@...ata.bg>,
	"Maciej W. Rozycki" <macro@...ux-mips.org>
Subject: Re: [PATCH] avoid negative shifts in radix-tree.c, take 2

Peter Lund <firefly@...64.dk> writes:

> Shifting by more than the width of the value on the left is also not allowed.

Shifting by the width of the value is not allowed as well.

> --- linux-2.6.22/lib/radix-tree.c.orig	2007-08-27 15:42:37.000000000 +0200
> +++ linux-2.6.22/lib/radix-tree.c	2007-08-29 13:19:19.000000000 +0200
> @@ -980,12 +980,13 @@
>  
>  static __init unsigned long __maxindex(unsigned int height)
>  {
> -	unsigned int tmp = height * RADIX_TREE_MAP_SHIFT;
> -	unsigned long index = (~0UL >> (RADIX_TREE_INDEX_BITS - tmp - 1)) >> 1;
> +	unsigned int width = height * RADIX_TREE_MAP_SHIFT;
> +	int          shift = RADIX_TREE_INDEX_BITS - width;
>  
> -	if (tmp >= RADIX_TREE_INDEX_BITS)
> -		index = ~0UL;
> -	return index;
> +	if (shift < 0)
> +		return ~0UL;
> +	else
> +		return ~0UL >> shift;

Since height can be zero, you still have undefined behaviour.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@...e.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/