[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20240813-fix_fencei_optimization-v1-1-2aadc2cdde95@rivosinc.com>
Date: Tue, 13 Aug 2024 16:02:17 -0700
From: Charlie Jenkins <charlie@...osinc.com>
To: Paul Walmsley <paul.walmsley@...ive.com>,
Palmer Dabbelt <palmer@...belt.com>, Albert Ou <aou@...s.berkeley.edu>,
Alexandre Ghiti <alexghiti@...osinc.com>, Atish Patra <atishp@...osinc.com>,
Samuel Holland <samuel.holland@...ive.com>,
Andrea Parri <parri.andrea@...il.com>
Cc: Palmer Dabbelt <palmer@...osinc.com>, linux-riscv@...ts.infradead.org,
linux-kernel@...r.kernel.org, Charlie Jenkins <charlie@...osinc.com>
Subject: [PATCH 1/2] riscv: Disable preemption while handling
PR_RISCV_CTX_SW_FENCEI_OFF
The icache will be flushed in switch_to() if force_icache_flush is true,
or in flush_icache_deferred() if icache_stale_mask is set. Between
setting force_icache_flush to false and calculating the new
icache_stale_mask, preemption needs to be disabled. There are two
reasons for this:
1. If CPU migration happens between force_icache_flush = false, and the
icache_stale_mask is set, an icache flush will not be emitted.
2. smp_processor_id() is used in set_icache_stale_mask() to mark the
current CPU as not needing another flush since a flush will have
happened either by userspace or by the kernel when performing the
migration. smp_processor_id() is currently called twice with preemption
enabled which causes a race condition. It allows
icache_stale_mask to be populated with inconsistent CPU ids.
Resolve these two issues by setting the icache_stale_mask before setting
force_icache_flush to false, and using get_cpu()/put_cpu() to obtain the
smp_processor_id().
Signed-off-by: Charlie Jenkins <charlie@...osinc.com>
Fixes: 6b9391b581fd ("riscv: Include riscv_set_icache_flush_ctx prctl")
---
arch/riscv/mm/cacheflush.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c
index a03c994eed3b..b81672729887 100644
--- a/arch/riscv/mm/cacheflush.c
+++ b/arch/riscv/mm/cacheflush.c
@@ -158,6 +158,7 @@ void __init riscv_init_cbo_blocksizes(void)
#ifdef CONFIG_SMP
static void set_icache_stale_mask(void)
{
+ int cpu = get_cpu();
cpumask_t *mask;
bool stale_cpu;
@@ -168,10 +169,11 @@ static void set_icache_stale_mask(void)
* concurrently on different harts.
*/
mask = ¤t->mm->context.icache_stale_mask;
- stale_cpu = cpumask_test_cpu(smp_processor_id(), mask);
+ stale_cpu = cpumask_test_cpu(cpu, mask);
cpumask_setall(mask);
- cpumask_assign_cpu(smp_processor_id(), mask, stale_cpu);
+ cpumask_assign_cpu(cpu, mask, stale_cpu);
+ put_cpu();
}
#endif
@@ -239,14 +241,12 @@ int riscv_set_icache_flush_ctx(unsigned long ctx, unsigned long scope)
case PR_RISCV_CTX_SW_FENCEI_OFF:
switch (scope) {
case PR_RISCV_SCOPE_PER_PROCESS:
- current->mm->context.force_icache_flush = false;
-
set_icache_stale_mask();
+ current->mm->context.force_icache_flush = false;
break;
case PR_RISCV_SCOPE_PER_THREAD:
- current->thread.force_icache_flush = false;
-
set_icache_stale_mask();
+ current->thread.force_icache_flush = false;
break;
default:
return -EINVAL;
--
2.45.0
Powered by blists - more mailing lists