linux-kernel - [tip:tracing/core] function-graph/x86: Replace unbalanced ret with jmp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <tip-194ec34184869f0de1cf255c924fc5299e1b3d27@git.kernel.org>
Date:	Wed, 14 Oct 2009 06:58:41 GMT
From:	tip-bot for Steven Rostedt <srostedt@...hat.com>
To:	linux-tip-commits@...r.kernel.org
Cc:	linux-kernel@...r.kernel.org, mathieu.desnoyers@...ymtl.ca,
	hpa@...or.com, mingo@...hat.com, fweisbec@...il.com,
	rostedt@...dmis.org, srostedt@...hat.com, tglx@...utronix.de,
	mingo@...e.hu
Subject: [tip:tracing/core] function-graph/x86: Replace unbalanced ret with jmp

Commit-ID:  194ec34184869f0de1cf255c924fc5299e1b3d27
Gitweb:     http://git.kernel.org/tip/194ec34184869f0de1cf255c924fc5299e1b3d27
Author:     Steven Rostedt <srostedt@...hat.com>
AuthorDate: Tue, 13 Oct 2009 16:33:50 -0400
Committer:  Ingo Molnar <mingo@...e.hu>
CommitDate: Wed, 14 Oct 2009 08:13:53 +0200

function-graph/x86: Replace unbalanced ret with jmp

The function graph tracer replaces the return address with a hook
to trace the exit of the function call. This hook will finish by
returning to the real location the function should return to.

But the current implementation uses a ret to jump to the real
return location. This causes a imbalance between calls and ret.
That is the original function does a call, the ret goes to the
handler and then the handler does a ret without a matching call.

Although the function graph tracer itself still breaks the branch
predictor by replacing the original ret, by using a second ret and
causing an imbalance, it breaks the predictor even more.

This patch replaces the ret with a jmp to keep the calls and ret
balanced. I tested this on one box and it showed a 1.7% increase in
performance. Another box only showed a small 0.3% increase. But no
box that I tested this on showed a decrease in performance by
making this change.

Signed-off-by: Steven Rostedt <rostedt@...dmis.org>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@...ymtl.ca>
Cc: Frederic Weisbecker <fweisbec@...il.com>
LKML-Reference: <20091013203425.042034383@...dmis.org>
Signed-off-by: Ingo Molnar <mingo@...e.hu>
---
 arch/x86/kernel/entry_32.S |    7 ++-----
 arch/x86/kernel/entry_64.S |    6 +++---
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
index c097e7d..7d52e9d 100644
--- a/arch/x86/kernel/entry_32.S
+++ b/arch/x86/kernel/entry_32.S
@@ -1185,17 +1185,14 @@ END(ftrace_graph_caller)
 
 .globl return_to_handler
 return_to_handler:
-	pushl $0
 	pushl %eax
-	pushl %ecx
 	pushl %edx
 	movl %ebp, %eax
 	call ftrace_return_to_handler
-	movl %eax, 0xc(%esp)
+	movl %eax, %ecx
 	popl %edx
-	popl %ecx
 	popl %eax
-	ret
+	jmp *%ecx
 #endif
 
 .section .rodata,"a"
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
index b5c061f..bd5bbdd 100644
--- a/arch/x86/kernel/entry_64.S
+++ b/arch/x86/kernel/entry_64.S
@@ -155,11 +155,11 @@ GLOBAL(return_to_handler)
 
 	call ftrace_return_to_handler
 
-	movq %rax, 16(%rsp)
+	movq %rax, %rdi
 	movq 8(%rsp), %rdx
 	movq (%rsp), %rax
-	addq $16, %rsp
-	retq
+	addq $24, %rsp
+	jmp *%rdi
 #endif
 
 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/