linux-kernel - [PATCH v2] blk-mq: Abort suspend when wakeup events are pending

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20251203-blkmq_skip_waiting-v2-1-aaf38fa5883d@oss.qualcomm.com>
Date: Wed, 03 Dec 2025 11:34:21 +0800
From: Cong Zhang <cong.zhang@....qualcomm.com>
To: Jens Axboe <axboe@...nel.dk>, Daniel Wagner <dwagner@...e.de>,
        Hannes Reinecke <hare@...e.de>, Ming Lei <ming.lei@...hat.com>
Cc: linux-arm-msm@...r.kernel.org, linux-block@...r.kernel.org,
        linux-kernel@...r.kernel.org, pavan.kondeti@....qualcomm.com,
        Cong Zhang <cong.zhang@....qualcomm.com>
Subject: [PATCH v2] blk-mq: Abort suspend when wakeup events are pending

During system suspend, wakeup capable IRQs for block device can be
delayed, which can cause blk_mq_hctx_notify_offline() to hang
indefinitely while waiting for pending request to complete.
Skip the request waiting loop and abort suspend when wakeup events are
pending to prevent the deadlock.

Fixes: bf0beec0607d ("blk-mq: drain I/O when all CPUs in a hctx are offline")
Signed-off-by: Cong Zhang <cong.zhang@....qualcomm.com>
---
The issue was found during system suspend with a no_soft_reset
virtio-blk device. Here is the detailed analysis:
- When system suspend starts and no_soft_reset is enabled, virtio-blk
  does not call its suspend callback.
- Some requests are dispatched and queued. After sending the virtqueue
  notifier, the kernel waits for an IRQ to complete the request.
- The virtio-blk IRQ is wakeup-capable. When the IRQ is triggered, it
  remains pending because the device is in the suspend process.
- While checking blk_mq_hctx_has_requests(), it detects that there are
  still pending requests.
- Since there is no way to complete these requests, the kernel gets
  stuck in the CPU hotplug thread.

We believe this could be a common issue. If the kernel enters the
blk_mq_hctx_has_requests() loop during suspend, wakeup-capable IRQs
cannot be processed, which can lead to a deadlock in this scenario.
This also improves the latency for wakup-capable IRQs. If a non-block
wakeup IRQ is pending, suspend is going to be abort anyway after this
step. Returning early avoids unnecessary delay and improve the suspend
latency.
---
Changes in v2:
- Add commmets for `if (pm_wakeup_pending)`
- Link to v1: https://lore.kernel.org/20251202-blkmq_skip_waiting-v1-1-f73d8a977ce0@oss.qualcomm.com
---
 block/blk-mq.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index d626d32f6e576f95bc68495c467a9d9c7b73a581..33a0062f9e56d3915b7d06f133548c16e1ca9aa6 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -23,6 +23,7 @@
 #include <linux/cache.h>
 #include <linux/sched/topology.h>
 #include <linux/sched/signal.h>
+#include <linux/suspend.h>
 #include <linux/delay.h>
 #include <linux/crash_dump.h>
 #include <linux/prefetch.h>
@@ -3707,6 +3708,7 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
 {
 	struct blk_mq_hw_ctx *hctx = hlist_entry_safe(node,
 			struct blk_mq_hw_ctx, cpuhp_online);
+	int ret = 0;
 
 	if (blk_mq_hctx_has_online_cpu(hctx, cpu))
 		return 0;
@@ -3727,12 +3729,24 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
 	 * frozen and there are no requests.
 	 */
 	if (percpu_ref_tryget(&hctx->queue->q_usage_counter)) {
-		while (blk_mq_hctx_has_requests(hctx))
+		while (blk_mq_hctx_has_requests(hctx)) {
+			/*
+			 * The wakeup capable IRQ handler of block device is
+			 * not called during suspend. Skip the loop by checking
+			 * pm_wakeup_pending to prevent the deadlock and improve
+			 * suspend latency.
+			 */
+			if (pm_wakeup_pending()) {
+				clear_bit(BLK_MQ_S_INACTIVE, &hctx->state);
+				ret = -EBUSY;
+				break;
+			}
 			msleep(5);
+		}
 		percpu_ref_put(&hctx->queue->q_usage_counter);
 	}
 
-	return 0;
+	return ret;
 }
 
 /*

---
base-commit: e538109ac71d801d26776af5f3c54f548296c29c
change-id: 20251128-blkmq_skip_waiting-732dab95acdb

Best regards,
-- 
Cong Zhang <cong.zhang@....qualcomm.com>