linux-kernel - Re: sched_rt_period_timer causing large latencies

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1522915882.5593.3.camel@gmx.de>
Date:   Thu, 05 Apr 2018 10:11:22 +0200
From:   Mike Galbraith <efault@....de>
To:     Nicholas Piggin <npiggin@...il.com>
Cc:     Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Steven Rostedt <rostedt@...dmis.org>
Subject: Re: sched_rt_period_timer causing large latencies

On Thu, 2018-04-05 at 17:44 +1000, Nicholas Piggin wrote:
> 
> > My method of dealing with the throttle beast from hell for ~big box RT
> > is to stomp it flat during boot, as otherwise jitter is awful.
> 
> How do you stomp it flat?

With a size 12 boot originally from SGI.  Their extra hairy beasts
tended to grind themselves to fine powder.

---
 Documentation/admin-guide/kernel-parameters.txt |    2 ++
 kernel/sched/core.c                             |   11 +++++++++++
 2 files changed, 13 insertions(+)

--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2788,6 +2788,8 @@
 	noresume	[SWSUSP] Disables resume and restores original swap
 			space.
 
+	nortsched       [KNL] Disables realtime task bandwidth control.
+
 	no-scroll	[VGA] Disables scrollback.
 			This is required for the Braillex ib80-piezo Braille
 			reader made by F.H. Papenmeier (Germany).
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7091,3 +7091,14 @@ const u32 sched_prio_to_wmult[40] = {
 };
 
 #undef CREATE_TRACE_POINTS
+
+/*
+ * Bootline option to disable sched_rt_runtime.
+ */
+static int __init parse_nortsched(char *arg)
+{
+	sysctl_sched_rt_runtime = -1;
+	return 0;
+}
+early_param("nortsched", parse_nortsched);
+

> If there could be some basic detection for throttling on a per-CPU
> basis before invoking the best from hell, would that be possible to
> move this out of normal paths?

It's one beast mauling the entire box, that's the problem.  I just
consider it a debugging tool, and turn it off (if it kicks in, the RT
game was already over).  RT disables RT group scheduling, so I don't
have to care about that evilness.

> Alternatively I think most/all of the RT runtime is coming from
> kernel threads, could they be exempted so this doesn't hit? Although
> it would be sad to suddenly inflict these latencies on someone who
> starts up an RT program :)

Yup, very sad.  It should probably become percpu.

	-Mike