linux-kernel - [PATCH v4 9/9] blk-mq: issue warning when offlining hctx with online isolcpus

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20241217-isolcpus-io-queues-v4-9-5d355fbb1e14@kernel.org>
Date: Tue, 17 Dec 2024 19:29:43 +0100
From: Daniel Wagner <wagi@...nel.org>
To: Jens Axboe <axboe@...nel.dk>, Keith Busch <kbusch@...nel.org>, 
 Christoph Hellwig <hch@....de>, Sagi Grimberg <sagi@...mberg.me>, 
 Kashyap Desai <kashyap.desai@...adcom.com>, 
 Sumit Saxena <sumit.saxena@...adcom.com>, 
 Shivasharan S <shivasharan.srikanteshwara@...adcom.com>, 
 Chandrakanth patil <chandrakanth.patil@...adcom.com>, 
 "Martin K. Petersen" <martin.petersen@...cle.com>, 
 Nilesh Javali <njavali@...vell.com>, GR-QLogic-Storage-Upstream@...vell.com, 
 Don Brace <don.brace@...rochip.com>, "Michael S. Tsirkin" <mst@...hat.com>, 
 Jason Wang <jasowang@...hat.com>, Paolo Bonzini <pbonzini@...hat.com>, 
 Stefan Hajnoczi <stefanha@...hat.com>, 
 Eugenio Pérez <eperezma@...hat.com>, 
 Xuan Zhuo <xuanzhuo@...ux.alibaba.com>, 
 Andrew Morton <akpm@...ux-foundation.org>, 
 Thomas Gleixner <tglx@...utronix.de>
Cc: Costa Shulyupin <costa.shul@...hat.com>, 
 Juri Lelli <juri.lelli@...hat.com>, 
 Valentin Schneider <vschneid@...hat.com>, Waiman Long <llong@...hat.com>, 
 Ming Lei <ming.lei@...hat.com>, 
 Michal Koutný <mkoutny@...e.com>, 
 Frederic Weisbecker <frederic@...nel.org>, Mel Gorman <mgorman@...e.de>, 
 Hannes Reinecke <hare@...e.de>, 
 Sridhar Balaraman <sbalaraman@...allelwireless.com>, 
 "brookxu.cn" <brookxu.cn@...il.com>, linux-kernel@...r.kernel.org, 
 linux-block@...r.kernel.org, linux-nvme@...ts.infradead.org, 
 megaraidlinux.pdl@...adcom.com, linux-scsi@...r.kernel.org, 
 storagedev@...rochip.com, virtualization@...ts.linux.dev, 
 Daniel Wagner <wagi@...nel.org>
Subject: [PATCH v4 9/9] blk-mq: issue warning when offlining hctx with
 online isolcpus

When we offlining a hardware context which also serves isolcpus mapped
to it, any IO issued by the isolcpus will stall as there is nothing
which handles the interrupts etc.

This configuration/setup is not supported at this point thus just issue
a warning.

Signed-off-by: Daniel Wagner <wagi@...nel.org>
---
 block/blk-mq.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index de15c0c76f874a2a863b05a23e0f3dba20cb6488..f9af0f5dd6aac8da855777acf2ffc61128f15a74 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3619,6 +3619,45 @@ static bool blk_mq_hctx_has_requests(struct blk_mq_hw_ctx *hctx)
 	return data.has_rq;
 }

+static void blk_mq_hctx_check_isolcpus_online(struct blk_mq_hw_ctx *hctx, unsigned int cpu)
+{
+	const struct cpumask *hk_mask;
+	int i;
+
+	if (!housekeeping_enabled(HK_TYPE_MANAGED_IRQ))
+		return;
+
+	hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
+
+	for (i = 0; i < hctx->nr_ctx; i++) {
+		struct blk_mq_ctx *ctx = hctx->ctxs[i];
+
+		if (ctx->cpu == cpu)
+			continue;
+
+		/*
+		 * Check if this context has at least one online
+		 * housekeeping CPU in this case the hardware context is
+		 * usable.
+		 */
+		if (cpumask_test_cpu(ctx->cpu, hk_mask) &&
+		    cpu_online(ctx->cpu))
+			break;
+
+		/*
+		 * The context doesn't have any online housekeeping CPUs
+		 * but there might be an online isolated CPU mapped to
+		 * it.
+		 */
+		if (cpu_is_offline(ctx->cpu))
+			continue;
+
+		pr_warn("%s: offlining hctx%d but there is still an online isolcpu CPU %d mapped to it, IO stalls expected\n",
+			hctx->queue->disk->disk_name,
+			hctx->queue_num, ctx->cpu);
+	}
+}
+
 static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx,
 		unsigned int this_cpu)
 {
@@ -3638,8 +3677,10 @@ static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx,
 			continue;

 		/* this hctx has at least one online CPU */
-		if (this_cpu != cpu)
+		if (this_cpu != cpu) {
+			blk_mq_hctx_check_isolcpus_online(hctx, this_cpu);
 			return true;
+		}
 	}

 	return false;

-- 
2.47.1