linux-kernel - Re: [interval_tree_test_init] BUG: soft lockup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Thu, 9 Nov 2017 08:17:15 -0800
From:   Davidlohr Bueso <dave@...olabs.net>
To:     Fengguang Wu <fengguang.wu@...el.com>, akpm@...ux-foundation.org
Cc:     Chris Wilson <chris@...is-wilson.co.uk>,
        linux-kernel@...r.kernel.org,
        Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [interval_tree_test_init] BUG: soft lockup - CPU#0 stuck for
 23s! [swapper/0:1]

On Wed, 08 Nov 2017, Fengguang Wu wrote:

>On Tue, Nov 07, 2017 at 08:58:14AM -0800, Davidlohr Bueso wrote:
>>On Tue, 07 Nov 2017, Fengguang Wu wrote:
>>
>>>[  265.102312] xz_dec_test: module loaded
>>>[  265.111774] xz_dec_test: Create a device node with 'mknod xz_dec_test c 246 0' and write .xz files to it.
>>>[  265.160320] atomic64_test: passed for x86-64 platform with CX8 and with SSE
>>>[  265.181966] glob: 64 self-tests passed, 0 failed
>>>[  265.196996] interval tree insert/remove
>>>[  292.348094] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [swapper/0:1]
>>>[  292.348094] irq event stamp: 6190340
>>>[  292.396418] hardirqs last  enabled at (6190339): [<ffffffff81fb006b>] restore_regs_and_iret+0x0/0x1d
>>>[  292.396418] hardirqs last disabled at (6190340): [<ffffffff81fb0cb8>] apic_timer_interrupt+0x98/0xb0
>>>[  292.396418] softirqs last  enabled at (6182242): [<ffffffff81fb2942>] __do_softirq+0x382/0x3f7
>>>[  292.396418] softirqs last disabled at (6182235): [<ffffffff810b6696>] irq_exit+0x51/0x7b
>>>[  292.396418] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.14.0-rc8 #40
>>>[  292.396418] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
>>>[  292.396418] task: ffff880072bf4040 task.stack: ffffc9000031c000
>>>[  292.396418] RIP: 0010:interval_tree_insert+0x4c/0xd4
>>
>>That would be:
>>
>>    for (i = 0; i < perf_loops; i++) {
>>	 for (j = 0; j < nnodes; j++)
>>		interval_tree_insert(nodes + j, &root);
>>	 for (j = 0; j < nnodes; j++)
>>		interval_tree_remove(nodes + j, &root);
>>    }
>>
>>Which if you are using the default params would end up doing insert() 10 million times.
>>How about decrementing perf_loops, it's pretty high.
>
>It works! Thank you!
>
>Tested-by: Fengguang Wu <fengguang.wu@...el.com>
>
>Fengguang
>
>>diff --git a/lib/interval_tree_test.c b/lib/interval_tree_test.c
>>index 0e343fd29570..1dbdf3704247 100644
>>--- a/lib/interval_tree_test.c
>>+++ b/lib/interval_tree_test.c
>>@@ -11,7 +11,7 @@
>>       MODULE_PARM_DESC(name, msg);
>>
>>__param(int, nnodes, 100, "Number of nodes in the interval tree");
>>-__param(int, perf_loops, 100000, "Number of iterations modifying the tree");
>>+__param(int, perf_loops, 1000, "Number of iterations modifying the tree");
>>
>>__param(int, nsearches, 100, "Number of searches to the interval tree");
>>__param(int, search_loops, 10000, "Number of iterations searching the tree");
>>
>>If that works I guess we'd have to limit the max values for such parameters
>>to avoid soft lockups. The same applies to your rbtree test report.

I've decided to not do this as different users might be ok with the risk,
just not the default.

Cc'ing akpm.

Thanks,
Davidlohr

--------8<--------------------------------------------------
[PATCH] lib/rbtree-test: lower default params

Fengguang reported soft lockups while running the rbtree
and interval tree test modules. The logic for these tests
all occur in init phase, and we currently are pounding
with the default values for number of nodes and number
of iterations of each test. Reduce the later by two
orders of magnitude. This does not influence the value
of the tests in that one thousand times by default is
enough to get the picture.

Reported-by: Fengguang Wu <fengguang.wu@...el.com>
Signed-off-by: Davidlohr Bueso <dbueso@...e.de>
---
 lib/interval_tree_test.c | 4 ++--
 lib/rbtree_test.c        | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/interval_tree_test.c b/lib/interval_tree_test.c
index 0e343fd29570..835242e74aaa 100644
--- a/lib/interval_tree_test.c
+++ b/lib/interval_tree_test.c
@@ -11,10 +11,10 @@
 	MODULE_PARM_DESC(name, msg);
 
 __param(int, nnodes, 100, "Number of nodes in the interval tree");
-__param(int, perf_loops, 100000, "Number of iterations modifying the tree");
+__param(int, perf_loops, 1000, "Number of iterations modifying the tree");
 
 __param(int, nsearches, 100, "Number of searches to the interval tree");
-__param(int, search_loops, 10000, "Number of iterations searching the tree");
+__param(int, search_loops, 1000, "Number of iterations searching the tree");
 __param(bool, search_all, false, "Searches will iterate all nodes in the tree");
 
 __param(uint, max_endpoint, ~0, "Largest value for the interval's endpoint");
diff --git a/lib/rbtree_test.c b/lib/rbtree_test.c
index 191a238e5a9d..7d36c1e27ff6 100644
--- a/lib/rbtree_test.c
+++ b/lib/rbtree_test.c
@@ -11,7 +11,7 @@
 	MODULE_PARM_DESC(name, msg);
 
 __param(int, nnodes, 100, "Number of nodes in the rb-tree");
-__param(int, perf_loops, 100000, "Number of iterations modifying the rb-tree");
+__param(int, perf_loops, 1000, "Number of iterations modifying the rb-tree");
 __param(int, check_loops, 100, "Number of iterations modifying and verifying the rb-tree");
 
 struct test_node {
-- 
2.13.6