Date:	Tue, 22 Mar 2016 14:53:28 +0200
From:	Nikolay Borisov <kernel@...p.com>
To:	viro@...iv.linux.org.uk
Cc:	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [RFC PATCH] dcache: give a chance to yield in shrink_dentry_list

A softlockup in shrink_dentry_list when called from shrink_dcache_sb
was observed on a very busy server. It's possible that the list
passed to shrink_dentry_list is so big that it takes a while to
dispose of all entries. Adding a simple cond_resched would give
the cpu a chance to do some other useful work and no trip the
softlockup watchdog.

Signed-off-by: Nikolay Borisov <kernel@...p.com>
---

Here is what the splat looked like. Even though this was observed
on a 3.12 kernel I don't see why it can't happen on recent 
kernels as well. 

[1294411.570734] BUG: soft lockup - CPU#3 stuck for 22s! [mount:33726]
[1294411.571005] Modules linked in: xt_comment ipip tunnel4 ip6_tunnel tunnel6 tcp_diag inet_diag xt_time act_police 
cls_basic sch_ingress hijack(O) netconsole xt_multiport xt_pkttype ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state veth 
openvswitch gre vxlan ip_tunnel xt_owner xt_conntrack iptable_mangle xt_nat iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 
nf_nat_ipv4 nf_nat xt_CT nf_conntrack iptable_raw ip6table_filter ip6_tables ipv6 ext2 dm_thin_pool dm_bio_prison
dm_persistent_data dm_bufio dm_mirror dm_region_hash dm_log ib_ipoib ib_cm ib_sa ib_mad ib_core ixgbe i2c_i801 lpc_ich 
mfd_core ioapic ioatdma igb dca i2c_algo_bit [last unloaded: ipmi_msghandler]

[1294411.571047] CPU: 3 PID: 33726 Comm: mount Tainted: G           O 3.12.52-clouder2 #1
[1294411.571048] Hardware name: Supermicro X9DRD-7LN4F(-JBOD)/X9DRD-EF/X9DRD-7LN4F, BIOS 3.0a 12/05/2013
[1294411.571050] task: ffff8833c55f9830 ti: ffff883fcfcf6000 task.ti: ffff883fcfcf6000
[1294411.571051] RIP: 0010:[<ffffffff811c2bc0>]  [<ffffffff811c2bc0>] __d_drop+0x60/0xf0
[1294411.571056] RSP: 0018:ffff883fcfcf7d38  EFLAGS: 00000206
[1294411.571057] RAX: ffffc9000b101770 RBX: ffff881fff8cc6a0 RCX: 0000000000000007
[1294411.571058] RDX: ffff883fcfcf6000 RSI: 0000000000000000 RDI: ffff8805bd5063c0
[1294411.571058] RBP: ffff883fcfcf7d38 R08: 0000000000000073 R09: 0000000000000000
[1294411.571059] R10: 7fffffffffffffff R11: 0000000000000000 R12: ffff8805bd506530
[1294411.571060] R13: ffff883fcfcf7cc8 R14: 0000000000000000 R15: ffff881fff8cc6a0
[1294411.571061] FS:  00007f0321cca7e0(0000) GS:ffff881fff8c0000(0000) knlGS:0000000000000000
[1294411.571062] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1294411.571063] CR2: 00007f0320d7e0c0 CR3: 0000001e050f6000 CR4: 00000000001407e0
[1294411.571063] Stack:
[1294411.571064]  ffff883fcfcf7d68 ffffffff811c3372 0000000000000202 ffff8805bd5063c0
[1294411.571068]  ffff8805bd506440 ffff881fd0c9ec00 ffff883fcfcf7dd8 ffffffff811c36e6
[1294411.571070]  0000000000000001 ffff883fcfcf7df8 ffff883fcfcf6000 ffff883fcfcf6000
[1294411.571073] Call Trace:
[1294411.571078]  [<ffffffff811c3372>] __dentry_kill+0x62/0x210
[1294411.571080]  [<ffffffff811c36e6>] shrink_dentry_list+0x1c6/0x390
[1294411.571082]  [<ffffffff811c3bcb>] shrink_dcache_sb+0xcb/0x110
[1294411.571086]  [<ffffffff811ad42a>] do_remount_sb+0x4a/0x190
[1294411.571089]  [<ffffffff811ce200>] do_mount+0xaf0/0xd20
[1294411.571091]  [<ffffffff811ce4c7>] SyS_mount+0x97/0xf0
[1294411.571095]  [<ffffffff8164c4b2>] system_call_fastpath+0x16/0x1b


 fs/dcache.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fs/dcache.c b/fs/dcache.c
index 2398f9f94337..0db355a50bc5 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -934,6 +934,9 @@ static void shrink_dentry_list(struct list_head *list)
 
 	while (!list_empty(list)) {
 		struct inode *inode;
+
+		cond_resched();
+
 		dentry = list_entry(list->prev, struct dentry, d_lru);
 		spin_lock(&dentry->d_lock);
 		parent = lock_parent(dentry);
-- 
2.5.0

Powered by blists - more mailing lists