linux-kernel - Re: [lkp] [fs] df4c0e36f1: NMI watchdog: BUG: soft lockup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Tue, 3 Nov 2015 00:29:36 +0300
From:	Andrey Ryabinin <ryabinin.a.a@...il.com>
To:	Dave Hansen <dave.hansen@...el.com>
Cc:	kernel test robot <ying.huang@...ux.intel.com>, LKP <lkp@...org>,
	LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	David Rientjes <rientjes@...gle.com>,
	Pekka Enberg <penberg@...nel.org>,
	"H. Peter Anvin" <hpa@...or.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...e.hu>, Andi Kleen <andi@...stfloor.org>,
	Joonsoo Kim <iamjoonsoo.kim@....com>,
	Christoph Lameter <cl@...ux.com>,
	Sasha Levin <sasha.levin@...cle.com>,
	Konstantin Khlebnikov <koct9i@...il.com>,
	Yuri Gribov <tetra2005@...il.com>,
	Andrey Konovalov <adech.fo@...il.com>,
	Konstantin Serebryany <kcc@...gle.com>,
	Dmitry Vyukov <dvyukov@...gle.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [lkp] [fs] df4c0e36f1: NMI watchdog: BUG: soft lockup - CPU#0
 stuck for 22s! [swapper/0:1]

2015-11-02 23:07 GMT+03:00 Dave Hansen <dave.hansen@...el.com>:
> On 11/02/2015 11:34 AM, Andrey Ryabinin wrote:
>>>> >>
>>>> >> [    1.159450] augmented rbtree testing -> 23675 cycles
>>>> >> [    1.864996]
>>>> >> It took less than a second, meanwhile in your case it didn't finish in
>>>> >> 22 seconds.
>>>> >>
>>>> >> This makes me think that your host is overloaded and the problem is on
>>>> >> your side.
>>> >
>>> > It's probably just a matter of putting some cond_resched()s in the test
>>> > code.
>> Yes, but is it worthwhile? It's very likely that lockup will just
>> trigger in another place.
>
> I'm guessing that the lockup here was because the tests were running for
> too long.  If we cond_resched() in there often enough, the kernel won't
> detect a softlockup at all.

Sure, but why are these tests running so long?
In my setup it takes less than a second to finish these tests.
On the same kernel version and config of course. Although I might have more
powerful hardware it doesn't explain such huge difference.
So these tests are actually fast tests. I guess that  the host is
overloaded and KVM guest runs
so slow that even these simple tests start triggering softlockup.

> It won't shift somewhere else.

That's not what I mean. Sure, the cond_resched() in rbtree_test_init()
will fix this particular softlockup.
But if even such normally fast tests now are running too long, then a
lot of other kernel code, which normally
runs fast, likely becomes too slow on Ying's setup and will trigger
another softlockup.
rbtree_test_init() is just the first such place.
In that case, sticking cond_resched()s across the whole kernel is not
a solution.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/