Date:	Thu, 6 Jun 2013 21:25:22 +0800
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	Stephen Rothwell <sfr@...b.auug.org.au>
Cc:	Davidlohr Bueso <davidlohr.bueso@...com>,
	linux-kernel@...r.kernel.org
Subject: [IPC] INFO: suspicious RCU usage.

Greetings,

I got the below dmesg and the first bad commit is

commit 1f6587114a689a5d7fdfb0d4abc818117e3182a5
Author: Davidlohr Bueso <davidlohr.bueso@...com>
Date:   Thu Jun 6 10:41:56 2013 +1000

    ipc: move rcu lock out of ipc_addid
    
    This patchset continues the work that began in the sysv ipc semaphore
    scaling series: https://lkml.org/lkml/2013/3/20/546
    
    Just like semaphores used to be, sysv shared memory and msg queues also
    abuse the ipc lock, unnecessarily holding it for operations such as
    permission and security checks.  This patchset mostly deals with mqueues,
    and while shared mem can be done in a very similar way, I want to get
    these patches out in the open first.  It also does some pending cleanups,
    mostly focused on the two level locking we have in ipc code, taking care
    of ipc_addid() and ipcctl_pre_down_nolock() - yes there are still
    functions that need to be updated as well.
    
    This patch:
    
    Make all callers explicitly take and release the RCU read lock.
    
    This addresses the two level locking seen in newary(), newseg() and
    newqueue().  For the last two, explicitly unlock the ipc object and the
    rcu lock, instead of calling the custom shm_unlock and msg_unlock
    functions.  The next patch will deal with the open coded locking for
    ->perm.lock
    
    Signed-off-by: Davidlohr Bueso <davidlohr.bueso@...com>
    Cc: Andi Kleen <andi@...stfloor.org>
    Cc: Rik van Riel <riel@...hat.com>
    Signed-off-by: Andrew Morton <akpm@...ux-foundation.org>

[   51.524946] 
[   51.525983] ===============================
[   51.532875] [ INFO: suspicious RCU usage. ]
[   51.535385] 3.10.0-rc4-next-20130606 #6 Not tainted
[   51.538304] -------------------------------
[   51.540937] /c/kernel-tests/src/stable/include/linux/rcupdate.h:471 Illegal context switch in RCU read-side critical section!
[   51.548110] 
[   51.548110] other info that might help us debug this:
[   51.548110] 
[   51.553055] 
[   51.553055] rcu_scheduler_active = 1, debug_locks = 1
[   51.557199] 2 locks held by trinity/1107:
[   51.560168]  #0:  (&ids->rw_mutex){+.+.+.}, at: [<ffffffff811e15ee>] ipcget+0x38/0x2b3
[   51.566465]  #1:  (rcu_read_lock){.+.+..}, at: [<ffffffff811e7698>] newseg+0x19d/0x3fd
[   51.572413] 
[   51.572413] stack backtrace:
[   51.574761] CPU: 0 PID: 1107 Comm: trinity Not tainted 3.10.0-rc4-next-20130606 #6
[   51.579331] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[   51.583068]  0000000000000001 ffff880004a07d88 ffffffff817b1f5c ffff880004a07db8
[   51.592119]  ffffffff810f2f1d ffffffff81b78569 00000000000001a8 0000000000000000
[   51.596726]  0000000000000000 ffff880004a07de8 ffffffff810ded5e ffff880004a07fd8
[   51.605189] Call Trace:
[   51.606409]  [<ffffffff817b1f5c>] dump_stack+0x19/0x1b
[   51.609632]  [<ffffffff810f2f1d>] lockdep_rcu_suspicious+0xeb/0xf4
[   51.612905]  [<ffffffff810ded5e>] __might_sleep+0x59/0x1dc
[   51.618614]  [<ffffffff81238623>] idr_preload+0x9b/0x142
[   51.621939]  [<ffffffff811e0e56>] ipc_addid+0x3d/0x193
[   51.624373]  [<ffffffff811e771c>] newseg+0x221/0x3fd
[   51.626596]  [<ffffffff811e7698>] ? newseg+0x19d/0x3fd
[   51.630177]  [<ffffffff811e1774>] ipcget+0x1be/0x2b3
[   51.633174]  [<ffffffff817bc094>] ? retint_swapgs+0x13/0x1b
[   51.636356]  [<ffffffff811e7a5a>] SyS_shmget+0x59/0x5d
[   51.639576]  [<ffffffff811e74fb>] ? shm_try_destroy_orphaned+0xbf/0xbf
[   51.643673]  [<ffffffff811e6ce5>] ? shm_get_unmapped_area+0x20/0x20
[   51.647321]  [<ffffffff811e6cf0>] ? shm_security+0xb/0xb
[   51.650831]  [<ffffffff817bcb27>] system_call_fastpath+0x16/0x1b

git bisect start 4e1e7059d375482daeeda395bba2939679b1ee14 e3e160d1c8b68beede8e47b281bf6369b833f1c5 --
git bisect good 04eb3039eba693d510952e8867fc0b057955f840  #    10  2013-06-06 19:18:45  shrinker: convert remaining shrinkers to count/scan API
git bisect good 870ce17d96260246bd343f55b55b6adb468aab76  #    10  2013-06-06 19:46:15  rtc: rtc-mpc5121: use devm_*() functions
git bisect good e75aff7f353f503d38863fd0a3e1d1e4a3312325  #    10  2013-06-06 19:52:37  x86: kill TIF_DEBUG
git bisect  bad 93f94fca3e3edc76bd3f392edcd8ddc13629c626  #     0  2013-06-06 19:58:13  ipc/sem.c: rename try_atomic_semop() to perform_atomic_semop(), docu update
git bisect good e6391c3a76e29b92a6c0183f4d94776c1ca9ecbc  #    10  2013-06-06 20:45:47  ia64: remove setting for saved_max_pfn
git bisect  bad 47cde6c8674ec0598b6da4c0813ec984ee209d97  #     0  2013-06-06 20:50:16  ipc,msg: shorten critical region in msgctl_down
git bisect good 343ff671539e6ce9bc849d881fcec5c004342e0a  #    10  2013-06-06 20:55:29  ipc/shmc.c: eliminate ugly 80-col tricks
git bisect  bad e3a7780fbb54d93fd053182d4b1f8c5596b37247  #     0  2013-06-06 21:02:39  ipc: introduce ipc object locking helpers
git bisect  bad 1f6587114a689a5d7fdfb0d4abc818117e3182a5  #     0  2013-06-06 21:12:01  ipc: move rcu lock out of ipc_addid
git bisect good 343ff671539e6ce9bc849d881fcec5c004342e0a  #    30  2013-06-06 21:15:37  ipc/shmc.c: eliminate ugly 80-col tricks
git bisect  bad 4e1e7059d375482daeeda395bba2939679b1ee14  #     0  2013-06-06 21:15:40  Add linux-next specific files for 20130606
git bisect  bad 4e1e7059d375482daeeda395bba2939679b1ee14  #     0  2013-06-06 21:15:52  Add linux-next specific files for 20130606

Thanks,
Fengguang

View attachment "dmesg-kvm-cairo-52292-20130606163350-3.10.0-rc4-next-20130606-6" of type "text/plain" (113396 bytes)

Download attachment "4e1e7059d375482daeeda395bba2939679b1ee14-bisect.log" of type "application/octet-stream" (13294 bytes)

View attachment ".config-bisect" of type "text/plain" (77628 bytes)

Powered by blists - more mailing lists