linux-kernel - Re: [BUG] kernel freezes with latest tree

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120111161431.GA1233@elte.hu>
Date:	Wed, 11 Jan 2012 17:14:32 +0100
From:	Ingo Molnar <mingo@...e.hu>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	David Ahern <dsahern@...il.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Eric Dumazet <eric.dumazet@...il.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Martin Schwidefsky <schwidefsky@...ibm.com>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Suresh Siddha <suresh.b.siddha@...el.com>
Subject: Re: [BUG] kernel freezes with latest tree


* Peter Zijlstra <peterz@...radead.org> wrote:

> On Wed, 2012-01-11 at 16:56 +0100, Ingo Molnar wrote:
> 
> > Well, what happens if every CPU runs load_balance() and we keep 
> > triggering:
> > 
> >                 if (loops++ > sysctl_sched_nr_migrate) {
> >                         *lb_flags |= LBF_NEED_BREAK;
> >                         break;
> >                 }
> > 
> > in this case load_balance() will do the retry:
> > 
> >                 if (lb_flags & LBF_NEED_BREAK) {
> >                         lb_flags &= ~LBF_NEED_BREAK;
> >                         goto redo;
> >                 }
> > 
> > but the retry starts the loop again:
> > 
> >         list_for_each_entry_safe(p, n, &busiest_cfs_rq->tasks, se.group_node) {
> > 
> > so nobody is able to make progress: livelock/lockup.
> 
> Ah, right! Silly me. One possibility is to rotate that list, except that
> won't work for the cgroup case where we have another iteration.
> 
> OK, here's an updated patch.. 
> 
> ---
> Subject: sched: Limit load-balance retries on lock-break
> From: Peter Zijlstra <a.p.zijlstra@...llo.nl>
> Date: Wed Jan 11 13:11:12 CET 2012
> 
> Eric and David reported dead machines and traced it to commit a195f004 ("sched:
> Fix load-balance lock-breaking"), it turns out there's still a
> scenario where we can end up re-trying forever.
> 
> Since there is no strict forward progress guarantee in the
> load-balance iteration we can get stuck re-retrying the same task-set
> over and over.
> 
> Creating a forward progress guarantee with the existing structure is
> somewhat non-trivial, for now simply terminate the retry loop after a
> few tries.
> 
> Reported-by: Eric Dumazet <eric.dumazet@...il.com>
> Reported-by: David Ahern <dsahern@...il.com>
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@...llo.nl>
> [eric: logic cleanup]
> Tested-by: Eric Dumazet <eric.dumazet@...il.com>
> Link: http://lkml.kernel.org/n/tip-ya9m8grb9wfc26uqnviq2wjq@git.kernel.org
> ---
>  kernel/sched/fair.c |   10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)

Thanks Peter, i'll get this fix to Linus ASAP.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/