linux-kernel - Regression in performance when using PREEMPT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20251226110249.0c2f1d8d@gandalf.local.home>
Date: Fri, 26 Dec 2025 11:02:49 -0500
From: Steven Rostedt <rostedt@...dmis.org>
To: LKML <linux-kernel@...r.kernel.org>, linux-rt-devel@...ts.linux.dev
Cc: Sebastian Andrzej Siewior <bigeasy@...utronix.de>, liangjlee@...gle.com
Subject: Regression in performance when using PREEMPT_RT

Hi Sebastian,

We are doing some experiments in running Android Pixel with a PREEMPT_RT
kernel, we found a few unacceptable performance regressions. One was in the
block layer. In non-rt, the ufshcd interrupt would trigger the BLOCK
softirq on another CPU. In RT, it always triggered the softirq on the same
CPU as the interrupt. As the interrupt line always triggers on CPU0, it
forces the BLOCK softirq to also always run on CPU 0, which is bad because
CPU 0 is a little core and the main work should be running on a big core.

In block/blk-mq.c:blk_mq_complete_need_ipi() there's this code:

	/*
	 * With force threaded interrupts enabled, raising softirq from an SMP
	 * function call will always result in waking the ksoftirqd thread.
	 * This is probably worse than completing the request on a different
	 * cache domain.
	 */
	if (force_irqthreads())
		return false;

When I saw "probably worse", I'm thinking this was decided by analysis and
not by any real numbers. Was it?

When we commented out the above if statement so that it did not return
false, things sped up to almost non-rt speeds again.

The fio benchmark went from 76MB to 94MB (higher is better). It's still not
at the level of non-rt, but this was definitely one of the areas that
caused the regression.

Is that exit out of the function truly needed?

Thanks,

-- Steve