| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Thu, 7 Mar 2013 22:12:24 +0800
From: Fengguang Wu <fengguang.wu@...el.com>
To: fengguang.wu@...el.com, Nathan Zimmer <nzimmer@....com>
Cc: Andrew Morton <akpm@...ux-foundation.org>,
Stephen Rothwell <sfr@...b.auug.org.au>,
linux-kernel@...r.kernel.org
Subject: [procfs] BUG: sleeping function called from invalid context at
mm/slub.c:925
Greetings,
I got the below dmesg and the first bad commit is
commit a21813be23329e2788164eab532e79cb0e513cfc
Author: Nathan Zimmer <nzimmer@....com>
Date: Sat Mar 2 15:25:37 2013 +1100
procfs: improve scaling in proc
I am currently tracking a hotlock reported by a customer on a large
system, 512 cores. I am currently running 3.8-rc7 but the issue looks
like it has been this way for a very long time. The offending lock is
proc_dir_entry->pde_unload_lock.
This patch converts the lock to use rcu. However the pde_openers list
still is controlled by a spin lock. I tested on a 4096 machine and the
lock doesn't seem hot at least according to perf.
This is a refresh of what was orignally suggested by Eric Dumazet some
time ago. I have also taken in some comments from Andrew and several
other people whose names escape me but I am quite grateful too.
Supporting numbers, lower is better, they are from the test I posted earlier.
cpuinfo baseline Rcu
tasks read-sec read-sec
1 0.0141 0.0141
2 0.0140 0.0142
4 0.0140 0.0141
8 0.0145 0.0140
16 0.0553 0.0168
32 0.1688 0.0549
64 0.5017 0.1690
128 1.7005 0.5038
256 5.2513 2.0804
512 8.0529 3.0162
Signed-off-by: Nathan Zimmer <nzimmer@....com>
Cc: "Eric W. Biederman" <ebiederm@...ssion.com>
Cc: Eric Dumazet <eric.dumazet@...il.com>
Cc: Alexander Viro <viro@...iv.linux.org.uk>
Cc: David Woodhouse <dwmw2@...radead.org>
Cc: Alexey Dobriyan <adobriyan@...il.com>
Cc: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Signed-off-by: Andrew Morton <akpm@...ux-foundation.org>
[ 78.994163] atkbd: probe of serio1 rejects match -19
[ 79.196148] Freeing unused kernel memory: 664k freed
[ 80.786460] BUG: sleeping function called from invalid context at /c/kernel-tests/src/tip/mm/slub.c:925
[ 80.787886] in_atomic(): 0, irqs_disabled(): 0, pid: 29, name: modprobe
[ 80.789636] 2 locks held by modprobe/29:
[ 80.790343] #0: (rcu_read_lock){.+.+.+}, at: [<ffffffff8117347b>] rcu_read_lock+0x0/0xbc
[ 80.794346] #1: (rcu_read_lock){.+.+.+}, at: [<ffffffff8117347b>] rcu_read_lock+0x0/0xbc
[ 80.797089] Pid: 29, comm: modprobe Not tainted 3.9.0-rc1-01618-gdf2363d #223
[ 80.798113] Call Trace:
[ 80.798856] [<ffffffff8106fc60>] __might_sleep+0x266/0x275
[ 80.799830] [<ffffffff81146311>] ? single_open+0x30/0xb6
[ 80.801733] [<ffffffff811132fd>] slab_pre_alloc_hook.isra.48+0x42/0x4f
[ 80.802779] [<ffffffff81114f7b>] kmem_cache_alloc+0x2a/0xf6
[ 80.803746] [<ffffffff8117e899>] ? cmdline_proc_open+0x25/0x25
[ 80.805383] [<ffffffff81146311>] single_open+0x30/0xb6
[ 80.806281] [<ffffffff810c05df>] ? rcu_is_cpu_idle+0x42/0x8a
[ 80.807242] [<ffffffff8114556d>] ? seq_release_private+0x59/0x59
[ 80.808865] [<ffffffff8117e890>] cmdline_proc_open+0x1c/0x25
[ 80.809811] [<ffffffff811738e6>] proc_reg_open+0x128/0x1e3
[ 80.810684] [<ffffffff811737be>] ? proc_reg_release+0x246/0x246
[ 80.811602] [<ffffffff81119d82>] do_dentry_open+0x228/0x34a
[ 80.813260] [<ffffffff8112a226>] ? __inode_permission+0x115/0x12b
[ 80.814223] [<ffffffff81119f00>] finish_open+0x5c/0x7a
[ 80.815056] [<ffffffff8112b5bb>] do_last.isra.16+0xbcc/0xecb
[ 80.815945] [<ffffffff8112a2bd>] ? inode_permission+0x81/0x8a
[ 80.817660] [<ffffffff8112cfa6>] path_openat.isra.17+0xf0/0x494
[ 80.818623] [<ffffffff8112e7c6>] do_filp_open+0x3a/0xad
[ 80.819475] [<ffffffff8113eef6>] ? __alloc_fd+0x25e/0x277
[ 80.821142] [<ffffffff8111b4dd>] do_sys_open+0x91/0x17b
[ 80.822065] [<ffffffff8111b5f2>] SyS_open+0x2b/0x3b
[ 80.822873] [<ffffffff813d6f6b>] system_call_fastpath+0x1a/0x1f
[ 80.827403]
[ 80.827765] ================================================
git bisect start df2363d9ad7ffd43b5e86604c7a28f4adc6bd75e 725a7c316ea0d1329b630e31f82d0a2e762e0a65 --
git bisect good 62159550cf357e60632a7ac147b4c66b1e2b0630 # 18 2013-03-07 21:37:08 freezer: do not send a fake signal to a PF_DUMPCORE thread
git bisect bad bcbe478b5a2c1c5831dbe1c31d4b6a7d7438ccee # 0 2013-03-07 21:39:14 aio: kill struct aio_ring_info
git bisect bad 93102c910da3c99169b97ed03fdd53c64eadc284 # 0 2013-03-07 21:41:16 gadget: remove only user of aio retry
git bisect bad df2c264613410a2f59d4f13082207d5e4c0381fe # 0 2013-03-07 21:43:13 ipc: set EFAULT as default error in load_msg()
git bisect bad 1b716364563aa3fcd04eddea6fe08bd49574ec7f # 0 2013-03-07 21:44:39 kexec: Use min() and min_t() to simplify logic
git bisect bad a21813be23329e2788164eab532e79cb0e513cfc # 0 2013-03-07 21:46:14 procfs: improve scaling in proc
git bisect good c391a36c9b415f3276d1059594a635e66d3f5d7a # 12 2013-03-07 21:47:58 coredump: make wait_for_dump_helpers() freezable
git bisect good c391a36c9b415f3276d1059594a635e66d3f5d7a # 38 2013-03-07 21:49:41 coredump: make wait_for_dump_helpers() freezable
git bisect bad df2363d9ad7ffd43b5e86604c7a28f4adc6bd75e # 0 2013-03-07 21:51:39 kernel/sys.c: make prctl(PR_SET_MM) generally available
git bisect good 91ce46c0d79238fc84ee5f8c0936d4dcb66d1fd3 # 31 2013-03-07 21:54:23 Revert "procfs: improve scaling in proc"
git bisect bad 9edbffb58ae00067e264ef70d5141c1d85049029 # 0 2013-03-07 21:57:05 Add linux-next specific files for 20130307
Thanks,
Fengguang
View attachment "dmesg-kvm-ant-6398-2013-03-07-12-53-09-3.9.0-rc1-01618-gdf2363d-223" of type "text/plain" (49925 bytes)
View attachment "df2363d9ad7ffd43b5e86604c7a28f4adc6bd75e-bisect.log" of type "text/plain" (6711 bytes)
View attachment ".config-bisect" of type "text/plain" (58564 bytes)
Powered by blists - more mailing lists