linux-kernel - Re: Is kernel optimized with dead store removal?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <19334.22971.970220.245930@pilspetsen.it.uu.se>
Date:	Thu, 25 Feb 2010 12:06:35 +0100
From:	Mikael Pettersson <mikpe@...uu.se>
To:	Roel Kluin <roel.kluin@...il.com>
Cc:	"lkml" <linux-kernel@...r.kernel.org>,
	Herbert Xu <herbert@...dor.apana.org.au>,
	"David S. Miller" <davem@...emloft.net>,
	linux-crypto@...r.kernel.org
Subject: Re: Is kernel optimized with dead store removal?

Roel Kluin writes:
 > According to http://cwe.mitre.org/data/slices/2000.html#14 due to optimization
 > A call to memset() can be removed as a dead store when the buffer is not used
 > after its value is overwritten. Does this optimization also occur during
 > compilation of the Linux kernel? Then I think I may have found some
 > vulnerabilities. One is sha1_update() where memset(temp, 0, sizeof(temp)); may
 > be removed.

Any such dead store removal is up to the compiler and the lifetime
of the object being clobbered. For 'auto' objects the optimization
is certainly likely.

This is only a problem if the memory (a thread stack, say) is recycled
and leaked uninitialized to user-space, but such bugs are squashed
fairly quickly upon discovery.

(checking gcc-4.4.3)
It seems that memset((volatile void*)&some_local_var, 0, sizeof(...))
just provokes a warning about the invalid type of memset()'s first
parameter, but then still optimizes the operation away.

You might need to call an out-of-line helper function for this to work.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/