linux-kernel - Re: [PATCH v7 05/10] scsi: Use block layer helpers to constrain queue affinity

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <fdbc2ddc-830b-4727-88ae-5347fea8fca8@flourine.local>
Date: Fri, 4 Jul 2025 14:30:16 +0200
From: Daniel Wagner <dwagner@...e.de>
To: Hannes Reinecke <hare@...e.de>
Cc: Daniel Wagner <wagi@...nel.org>, Jens Axboe <axboe@...nel.dk>, 
	Keith Busch <kbusch@...nel.org>, Christoph Hellwig <hch@....de>, Sagi Grimberg <sagi@...mberg.me>, 
	"Michael S. Tsirkin" <mst@...hat.com>, Aaron Tomlin <atomlin@...mlin.com>, 
	"Martin K. Petersen" <martin.petersen@...cle.com>, Thomas Gleixner <tglx@...utronix.de>, 
	Costa Shulyupin <costa.shul@...hat.com>, Juri Lelli <juri.lelli@...hat.com>, 
	Valentin Schneider <vschneid@...hat.com>, Waiman Long <llong@...hat.com>, Ming Lei <ming.lei@...hat.com>, 
	Frederic Weisbecker <frederic@...nel.org>, Mel Gorman <mgorman@...e.de>, 
	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>, linux-kernel@...r.kernel.org, linux-block@...r.kernel.org, 
	linux-nvme@...ts.infradead.org, megaraidlinux.pdl@...adcom.com, linux-scsi@...r.kernel.org, 
	storagedev@...rochip.com, virtualization@...ts.linux.dev, 
	GR-QLogic-Storage-Upstream@...vell.com
Subject: Re: [PATCH v7 05/10] scsi: Use block layer helpers to constrain
 queue affinity

On Fri, Jul 04, 2025 at 12:28:49PM +0200, Hannes Reinecke wrote:
> It really shouldn't be an issue when the cpus are distributed 'correctly'
> :-)

If I get the drift, we start to discuss how the mapping could be
normally, so not for isolcpus. The isolcpus case is just how many hwq
are available (and affinity):

 num queues to map = min(num housekeeping CPUs, #hwq)

and then it's common code, no special housekeeping mapping code.

> We have several possibilities:
> -> #hwq > num_possible_cpus: easy, 1:1 mapping, no problem

I agree, no problem here.

> -> num_online_cpu < #hwq < num_possible_cpus: Not as easy, but if we
>    ensure that each online cpu is mapped to a different hwq we don't
>    have a performance impact.

This should also be fairly straightforward too. First assign each online
CPU a hwq and distribute the rest of the hwq amount the rest of the
possible offline CPUs.

> -> #hwq < num_online_cpu: If we ensure that a) the number of online cpus
>    per hwq is (roughly) identical we won't have a performance impact.
>    As a bonus we should strive to have the number of offline cpus
>    distributed equally on each hwq.

__group_cpus_evenly is handling this pretty well.

> Of course, that doesn't take into accound NUMA locality; with NUMA locality
> you would need to ensure to have at least one CPU per NUMA node
> mapped to each hwq. Which actually would impose a lower limit on the
> number (and granularity!) of hwqs (namely the number of NUMA nodes), but
> that's fair, I guess.

Again __group_cpus_evenly is taking NUMA into account as I understand it.

> But this really can be delegated to later patches; initially we really
> should identify which drivers might have issues with CPU hotplug,
> and at the very least issue a warning for these drivers.

There are different ways I suppose. My approach is not to change the
drivers too much because I don't have access to all the hardware for
testing. Instead extend the core code so that the different cases are
covered.