[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <235e470402ee86a6bf94cfe73543b0c0f5a42071.camel@gmx.de>
Date: Thu, 31 Oct 2024 11:21:34 +0100
From: Mike Galbraith <efault@....de>
To: LKML <linux-kernel@...r.kernel.org>
Cc: linux-mm <linux-mm@...ck.org>
Subject: RFC [PATCH1] mm: vmscan: Limit isolate_lru_folios() IRQ holdoff
latency impact
(was regression: mm: vmscan: - size XL irqoff time increase v6.10+)
While examining a service latency regression beginning at v6.10, it was
discovered that isolate_lru_folios() runtimes could cause IRQ holdoff
spanning several ticks. Worse measured was 78ms.
Given it's called with IRQs disabled, cap its runtme to around a ms.
With this patch, and one to mitigate move_expired_inodes() induced
wakeup latency hits up 66ms, the load that repeatably produced this:
(6.12.0-rc5 running osc kernel package build + ram sized bonnie)
T: 0 ( 4541) P:99 I:1000 C: 773124 Min: 1 Act: 6 Avg: 3 Max: 53381
T: 1 ( 4542) P:99 I:1500 C: 515381 Min: 1 Act: 7 Avg: 3 Max: 60119
T: 2 ( 4543) P:99 I:2000 C: 386465 Min: 1 Act: 5 Avg: 4 Max: 67469
T: 3 ( 4544) P:99 I:2500 C: 309182 Min: 1 Act: 7 Avg: 4 Max: 65534
T: 4 ( 4545) P:99 I:3000 C: 257682 Min: 1 Act: 6 Avg: 4 Max: 57776
T: 5 ( 4546) P:99 I:3500 C: 220874 Min: 1 Act: 4 Avg: 4 Max: 64737
T: 6 ( 4547) P:99 I:4000 C: 193289 Min: 1 Act: 6 Avg: 4 Max: 52567
T: 7 ( 4548) P:99 I:4500 C: 171799 Min: 1 Act: 6 Avg: 5 Max: 52606
latency went to this:
T: 0 ( 6078) P:99 I:1000 C:1031230 Min: 1 Act: 7 Avg: 4 Max: 4449
T: 1 ( 6079) P:99 I:1500 C: 687505 Min: 1 Act: 5 Avg: 4 Max: 2075
T: 2 ( 6080) P:99 I:2000 C: 515627 Min: 1 Act: 334 Avg: 5 Max: 2987
T: 3 ( 6081) P:99 I:2500 C: 412503 Min: 1 Act: 8 Avg: 5 Max: 2425
T: 4 ( 6082) P:99 I:3000 C: 343752 Min: 1 Act: 11 Avg: 5 Max: 1466
T: 5 ( 6083) P:99 I:3500 C: 294643 Min: 1 Act: 282 Avg: 5 Max: 3844
T: 6 ( 6084) P:99 I:4000 C: 257813 Min: 1 Act: 7 Avg: 5 Max: 3291
T: 7 ( 6085) P:99 I:4500 C: 229167 Min: 1 Act: 9 Avg: 5 Max: 3516
Signed-off-by: Mike Galbraith <efault@....de>
---
mm/vmscan.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -63,6 +63,9 @@
#include <linux/swapops.h>
#include <linux/balloon_compaction.h>
#include <linux/sched/sysctl.h>
+#ifndef CONFIG_PREEMPT_RT
+#include <linux/sched/clock.h>
+#endif
#include "internal.h"
#include "swap.h"
@@ -1657,6 +1660,10 @@ static unsigned long isolate_lru_folios(
unsigned long skipped = 0;
unsigned long scan, total_scan, nr_pages;
LIST_HEAD(folios_skipped);
+#ifndef CONFIG_PREEMPT_RT
+ u64 then = local_clock();
+ int batch = 0;
+#endif
total_scan = 0;
scan = 0;
@@ -1709,6 +1716,18 @@ static unsigned long isolate_lru_folios(
move_to = dst;
move:
list_move(&folio->lru, move_to);
+
+#ifndef CONFIG_PREEMPT_RT
+ /*
+ * IRQs are disabled, cap holdoff at a millisecond or so.
+ */
+ batch += nr_pages;
+ if (batch < 1024)
+ continue;
+ if (local_clock() - then >= NSEC_PER_MSEC)
+ break;
+ batch = 0;
+#endif
}
/*
View attachment "irqoff-trace-6.12.0.g4236f913-master" of type "text/plain" (2159 bytes)
Powered by blists - more mailing lists