linux-kernel - Re: [PATCH v6 6/9] isolation: introduce io

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aB1iolILQcvvHDE9@fedora>
Date: Fri, 9 May 2025 10:04:18 +0800
From: Ming Lei <ming.lei@...hat.com>
To: Daniel Wagner <dwagner@...e.de>
Cc: Hannes Reinecke <hare@...e.de>, Daniel Wagner <wagi@...nel.org>,
	Jens Axboe <axboe@...nel.dk>, Keith Busch <kbusch@...nel.org>,
	Christoph Hellwig <hch@....de>, Sagi Grimberg <sagi@...mberg.me>,
	"Michael S. Tsirkin" <mst@...hat.com>,
	"Martin K. Petersen" <martin.petersen@...cle.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Costa Shulyupin <costa.shul@...hat.com>,
	Juri Lelli <juri.lelli@...hat.com>,
	Valentin Schneider <vschneid@...hat.com>,
	Waiman Long <llong@...hat.com>,
	Frederic Weisbecker <frederic@...nel.org>,
	Mel Gorman <mgorman@...e.de>,
	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
	linux-kernel@...r.kernel.org, linux-block@...r.kernel.org,
	linux-nvme@...ts.infradead.org, megaraidlinux.pdl@...adcom.com,
	linux-scsi@...r.kernel.org, storagedev@...rochip.com,
	virtualization@...ts.linux.dev,
	GR-QLogic-Storage-Upstream@...vell.com
Subject: Re: [PATCH v6 6/9] isolation: introduce io_queue isolcpus type

On Fri, Apr 25, 2025 at 09:32:16AM +0200, Daniel Wagner wrote:
> On Fri, Apr 25, 2025 at 08:26:22AM +0200, Hannes Reinecke wrote:
> > On 4/24/25 20:19, Daniel Wagner wrote:
> > > Multiqueue drivers spreading IO queues on all CPUs for optimal
> > > performance. The drivers are not aware of the CPU isolated requirement
> > > and will spread all queues ignoring the isolcpus configuration.
> > > 
> > > Introduce a new isolcpus mask which allows the user to define on which
> > > CPUs IO queues should be placed. This is similar to the managed_irq but
> > > for drivers which do not use the managed IRQ infrastructure.
> > > 
> > > Signed-off-by: Daniel Wagner <wagi@...nel.org>
> > > ---
> > >   include/linux/sched/isolation.h | 1 +
> > >   kernel/sched/isolation.c        | 7 +++++++
> > >   2 files changed, 8 insertions(+)
> > > 
> > Reviewed-by: Hannes Reinecke <hare@...e.de>
> 
> Just realized I forgot to also add some document on this new argument:
> 
> 			io_queue
> 			  Isolate from IO queue work caused by multiqueue
> 			  device drivers. Restrict the placement of
> 			  queues to housekeeping CPUs only, ensuring that
> 			  all IO work is processed by a housekeeping CPU.
> 
> 			  Note: When an isolated CPU issues an IO, it is
> 			  forwarded to a housekeeping CPU. This will
> 			  trigger a software interrupt on the completion
> 			  path.
> 
> 			  Note: It is not possible to offline housekeeping
> 			  CPUs that serve isolated CPUs.

This patch adds kernel parameter only, but not apply it at all, the above
words just confuses everyone, so I'd suggest to not expose the kernel
command line & document until the whole mechanism is supported.

Especially 'irqaffinity=0 isolcpus=io_queue' requires the application
to offline CPU in order, which has to be documented:

https://lore.kernel.org/all/cc5e44dd-e1dc-4f24-88d9-ce45a8b0794f@flourine.local/

Thanks,
Ming