linux-kernel - [PATCH] sched: update_rq_clock() must skip ONE update

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 30 Mar 2014 09:24:04 +0200
From:	Mike Galbraith <umgwanakikbuti@...il.com>
To:	Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...e.hu>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>
Subject: [PATCH] sched: update_rq_clock() must skip ONE update

NEWSFLASH: Wimpy latent scheduler buglet turns axe-murderer...

Root cause being us staying in kernel for weeks at a time during boot:

Subverting ftrace, we see below that updates _can_ be skipped for AGES,
which can make very bad things happen.  It's supposed to skip ONE, as an
optimization. not lord knows how many.  Here we see we've been skipping
since 4915630446, resulting in the watchdog being credited with a 137ms
delta_exec.  Bogus, and..

        modprobe-134   [000]     5.027903: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.037857: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.037864: update_rq_clock <-enqueue_task
        modprobe-134   [000]     5.037866: enqueue_task_watchdog: WATCHDOG/0 enqueue 4915630446
        modprobe-134   [000]     5.044965: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.046275: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.050274: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.052862: update_rq_clock <-dequeue_task
        modprobe-134   [000]     5.052866: pick_next_task_watchdog: WATCHDOG/0 select 4915630446
      watchdog/0-7     [000]     5.052868: finish_task_switch: WATCHDOG/0 runs 5052867644
      watchdog/0-7     [000]     5.052869: watchdog: WATCHDOG/0 DELAYED 137238048 max: 137238048
      watchdog/0-7     [000]     5.052872: update_rq_clock <-dequeue_task
      watchdog/0-7     [000]     5.052873: dequeue_task_watchdog: WATCHDOG/0 dequeue 5052872018
      watchdog/0-7     [000]     5.052874: put_prev_task_watchdog: WATCHDOG/0 STOP trace : delta_exec 137241572 sum_exec 137254287

..delta_exec was sufficient in a NOPREEMPT kernel on zillion disk box
that disk discovery combined with swsusp wanting to find out if it needs
to resume, that the featherweight watchdog was credited with delta_exec
HUGE enough to be incapacitated by the rt throttle long enough for the
hard lockup detector to trigger.

Throttled watchdog is bad juju, led to -ENOBOOT for IO beast.

Cc: <stable@...r.kernel.org>
Signed-off-by: 	Mike Galbraith <umgwanakikbuti@...il.com>
---
 kernel/sched/core.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -118,7 +118,7 @@ void update_rq_clock(struct rq *rq)
 {
 	s64 delta;
 
-	if (rq->skip_clock_update > 0)
+	if (rq->skip_clock_update-- > 0)
 		return;
 
 	delta = sched_clock_cpu(cpu_of(rq)) - rq->clock;


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/