linux-kernel - [PATCH 4/6] perf: Optimize get_recursion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20201030151955.187580298@infradead.org>
Date:   Fri, 30 Oct 2020 16:13:49 +0100
From:   Peter Zijlstra <peterz@...radead.org>
To:     mingo@...nel.org, tglx@...utronix.de, rostedt@...dmis.org
Cc:     linux-kernel@...r.kernel.org, kan.liang@...ux.intel.com,
        acme@...nel.org, mark.rutland@....com,
        alexander.shishkin@...ux.intel.com, jolsa@...hat.com,
        namhyung@...nel.org, ak@...ux.intel.com, eranian@...gle.com,
        peterz@...radead.org, Jesper Dangaard Brouer <brouer@...hat.com>
Subject: [PATCH 4/6] perf: Optimize get_recursion_context()

  "Look ma, no branches!"

Cc: Jesper Dangaard Brouer <brouer@...hat.com>
Cc: Steven Rostedt <rostedt@...dmis.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
---
 kernel/events/internal.h |   17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

--- a/kernel/events/internal.h
+++ b/kernel/events/internal.h
@@ -205,16 +205,15 @@ DEFINE_OUTPUT_COPY(__output_copy_user, a
 
 static inline int get_recursion_context(int *recursion)
 {
-	int rctx;
+	unsigned int pc = preempt_count();
+	unsigned int rctx = 0;
 
-	if (unlikely(in_nmi()))
-		rctx = 3;
-	else if (in_irq())
-		rctx = 2;
-	else if (in_serving_softirq())
-		rctx = 1;
-	else
-		rctx = 0;
+	if (pc & (NMI_MASK))
+		rctx++;
+	if (pc & (NMI_MASK | HARDIRQ_MASK))
+		rctx++;
+	if (pc & (NMI_MASK | HARDIRQ_MASK | SOFTIRQ_OFFSET))
+		rctx++;
 
 	if (recursion[rctx])
 		return -1;