lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:	Sun, 30 Mar 2014 09:24:04 +0200
From:	Mike Galbraith <umgwanakikbuti@...il.com>
To:	Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...e.hu>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>
Subject: [PATCH] sched: update_rq_clock() must skip ONE update

NEWSFLASH: Wimpy latent scheduler buglet turns axe-murderer...

Root cause being us staying in kernel for weeks at a time during boot:

Subverting ftrace, we see below that updates _can_ be skipped for AGES,
which can make very bad things happen.  It's supposed to skip ONE, as an
optimization. not lord knows how many.  Here we see we've been skipping
since 4915630446, resulting in the watchdog being credited with a 137ms
delta_exec.  Bogus, and..

        modprobe-134   [000]     5.027903: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.037857: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.037864: update_rq_clock <-enqueue_task
        modprobe-134   [000]     5.037866: enqueue_task_watchdog: WATCHDOG/0 enqueue 4915630446
        modprobe-134   [000]     5.044965: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.046275: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.050274: update_rq_clock <-scheduler_tick
        modprobe-134   [000]     5.052862: update_rq_clock <-dequeue_task
        modprobe-134   [000]     5.052866: pick_next_task_watchdog: WATCHDOG/0 select 4915630446
      watchdog/0-7     [000]     5.052868: finish_task_switch: WATCHDOG/0 runs 5052867644
      watchdog/0-7     [000]     5.052869: watchdog: WATCHDOG/0 DELAYED 137238048 max: 137238048
      watchdog/0-7     [000]     5.052872: update_rq_clock <-dequeue_task
      watchdog/0-7     [000]     5.052873: dequeue_task_watchdog: WATCHDOG/0 dequeue 5052872018
      watchdog/0-7     [000]     5.052874: put_prev_task_watchdog: WATCHDOG/0 STOP trace : delta_exec 137241572 sum_exec 137254287

..delta_exec was sufficient in a NOPREEMPT kernel on zillion disk box
that disk discovery combined with swsusp wanting to find out if it needs
to resume, that the featherweight watchdog was credited with delta_exec
HUGE enough to be incapacitated by the rt throttle long enough for the
hard lockup detector to trigger.

Throttled watchdog is bad juju, led to -ENOBOOT for IO beast.

Cc: <stable@...r.kernel.org>
Signed-off-by: 	Mike Galbraith <umgwanakikbuti@...il.com>
---
 kernel/sched/core.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -118,7 +118,7 @@ void update_rq_clock(struct rq *rq)
 {
 	s64 delta;
 
-	if (rq->skip_clock_update > 0)
+	if (rq->skip_clock_update-- > 0)
 		return;
 
 	delta = sched_clock_cpu(cpu_of(rq)) - rq->clock;


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists