linux-kernel - Re: [nohz] 2a16fc93d2c: kernel lockup on idle injection

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 16 Dec 2014 15:32:28 +0100 (CET)
From:	Thomas Gleixner <tglx@...utronix.de>
To:	Preeti U Murthy <preeti@...ux.vnet.ibm.com>
cc:	Viresh Kumar <viresh.kumar@...aro.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Fengguang Wu <fengguang.wu@...el.com>,
	Frederic Weisbecker <frederic@...nel.org>,
	"Pan, Jacob jun" <jacob.jun.pan@...el.com>,
	LKML <linux-kernel@...r.kernel.org>, LKP <lkp@...org>,
	Peter Zijlstra <peterz@...radead.org>
Subject: Re: [nohz] 2a16fc93d2c: kernel lockup on idle injection

On Tue, 16 Dec 2014, Thomas Gleixner wrote:
> Now the powerclamp mess is a different story.
> 
> Calling tick_nohz_idle_enter()/exit() from outside the idle task is
> just broken. Period.
> 
> Trying to work around that madness in the core code is just fiddling
> with the symptoms and ignoring the root cause. And the root cause is
> simply that powerclamp calls tick_nohz_idle_enter()/exit().

The only sane solution for that insanity is to rip out the thread code
from powerclamp and replace it with a simple per cpu timer.

timerfn() 
{
   too_hot = magic();
   sched_fair_this_cpu_wreckaged(too_hot);   
   restart_clamp_timer();
}

And have the following in sched

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index df2cdf77f899..d257b62ee533 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4989,6 +4989,34 @@ preempt:
 		set_last_buddy(se);
 }
 
+#ifdef CONFIG_WRECKAGE_SCHED_FAIR
+static struct static_key wreckage_key = STATIC_KEY_INIT;
+static DEFINE_PER_CPU(cpu_is_wreckaged, bool);
+
+void sched_fair_this_cpu_wreckaged(bool wreckaged)
+{
+	this_cpu_write(sched_block_fair, wreckaged);
+}
+
+void sched_fair_cpu_wreckage_control(bool on)
+{
+	if (on)
+		static_key_slow_inc(&wreckage_key);
+	else
+		static_key_slow_dec(&wreckage_key);
+}
+
+static inline bool class_fair_disabled(void)
+{
+	if (static_key_false(&wreckage_key))
+		return false;
+	return this_cpu_read(sched_block_fair);
+}
+
+#else
+static inline bool class_fair_disabled(void) { return false; }
+#endif
+
 static struct task_struct *
 pick_next_task_fair(struct rq *rq, struct task_struct *prev)
 {
@@ -4997,6 +5025,8 @@ pick_next_task_fair(struct rq *rq, struct task_struct *prev)
 	struct task_struct *p;
 	int new_tasks;
 
+	if (class_fair_disabled())
+		goto idle;
 again:
 #ifdef CONFIG_FAIR_GROUP_SCHED
 	if (!cfs_rq->nr_running)

The static key is enabled once the powerclamp mess starts. So nobody
else than powerclamp users are affected by this and rightfully so.

Not pretty, but better than a gazillion workarounds all over the place
to make "pretending I'm idle" actually work. This is basically the
same mechanism as we have with the RT throttler, where a RT hog will
be put onto hold for some time. We just put all sched other tasks on
hold while still allowing RT tasks and everything else to work.

Thoughts?

Thanks,

	tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/