linux-hardening - [PATCH][V2] fortify: add branch hints on unlikely fortify

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250707132340.1768807-1-colin.i.king@gmail.com>
Date: Mon,  7 Jul 2025 14:23:40 +0100
From: Colin Ian King <colin.i.king@...il.com>
To: Kees Cook <kees@...nel.org>,
	linux-hardening@...r.kernel.org
Cc: kernel-janitors@...r.kernel.org,
	linux-kernel@...r.kernel.org
Subject: [PATCH][V2] fortify: add branch hints on unlikely fortify_panic paths

Analysis with gcov while running the stress-ng urandom stressor
shows that there are a couple of fortify panic paths that are highly
unlikely to be executed for well-behaving code. Adding appropriate
branch hints improves the stress-ng urandom stressor my a small
but statistically measureable amount. Ran 100 x 1 minute tests and
measured the stressor bogo-op rates on a Debian based Intel(R)
Core(TM) Ultra 9 285K with a 6.15 kernel with turbo disabled to
reduce jitter.

Results based on a Geometic Mean of 100 tests:

Without patch: 50512.95 bogo-ops/sec
With patch:    50819.58 bogo-ops/sec

%Std.Deviation of ~0.18%, so low jitter in results, improvement of ~0.6%

Branch hints can only be enabled if CONFIG_TRACE_BRANCH_PROFILING due
to a static variable being declared in __branch_check__ when using
trace branch profiling in the unlikely macro causing build issues.

Signed-off-by: Colin Ian King <colin.i.king@...il.com>
---
V2: add fortify_unlikely macro wrapper for CONFIG_TRACE_BRANCH_PROFILING
    build config.
---
 include/linux/fortify-string.h | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/include/linux/fortify-string.h b/include/linux/fortify-string.h
index e4ce1cae03bf..c740114bcbf8 100644
--- a/include/linux/fortify-string.h
+++ b/include/linux/fortify-string.h
@@ -10,6 +10,12 @@
 #define __FORTIFY_INLINE extern __always_inline __gnu_inline __overloadable
 #define __RENAME(x) __asm__(#x)
 
+#if defined(CONFIG_TRACE_BRANCH_PROFILING)
+#define fortify_unlikely(x)		(x)
+#else
+#define fortify_unlikely(x)		unlikely(x)
+#endif
+
 #define FORTIFY_REASON_DIR(r)		FIELD_GET(BIT(0), r)
 #define FORTIFY_REASON_FUNC(r)		FIELD_GET(GENMASK(7, 1), r)
 #define FORTIFY_REASON(func, write)	(FIELD_PREP(BIT(0), write) | \
@@ -593,9 +599,9 @@ __FORTIFY_INLINE bool fortify_memcpy_chk(__kernel_size_t size,
 	 * (The SIZE_MAX test is to optimize away checks where the buffer
 	 * lengths are unknown.)
 	 */
-	if (p_size != SIZE_MAX && p_size < size)
+	if (fortify_unlikely(p_size != SIZE_MAX && p_size < size))
 		fortify_panic(func, FORTIFY_WRITE, p_size, size, true);
-	else if (q_size != SIZE_MAX && q_size < size)
+	else if (fortify_unlikely(q_size != SIZE_MAX && q_size < size))
 		fortify_panic(func, FORTIFY_READ, p_size, size, true);
 
 	/*
-- 
2.50.0