[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20100226131931.GA19695@elte.hu>
Date: Fri, 26 Feb 2010 14:19:31 +0100
From: Ingo Molnar <mingo@...e.hu>
To: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Cc: linux-kernel@...r.kernel.org, laijs@...fujitsu.com,
dipankar@...ibm.com, akpm@...ux-foundation.org,
mathieu.desnoyers@...ymtl.ca, josh@...htriplett.org,
dvhltc@...ibm.com, niv@...ibm.com, tglx@...utronix.de,
peterz@...radead.org, rostedt@...dmis.org, Valdis.Kletnieks@...edu,
dhowells@...hat.com
Subject: Re: [PATCH tip/core/rcu 0/2] rcu: fixes for RCU lockdep
another lockdep splat:
[ 26.484222] PM: Adding info for No Bus:vcs10
[ 26.484520] PM: Adding info for No Bus:vcsa10
[ 297.312223] ------------[ cut here ]------------
[ 297.313012] WARNING: at kernel/softirq.c:143 local_bh_enable_ip+0x90/0xd0()
[ 297.313012] Hardware name: System Product Name
[ 297.313012] Modules linked in:
[ 297.313012] Pid: 0, comm: swapper Not tainted 2.6.33-tip-00954-g8f6d799-dirty #18817
[ 297.313012] Call Trace:
[ 297.313012] [<c103bf6d>] warn_slowpath_common+0x6d/0xa0
[ 297.313012] [<c1042880>] ? local_bh_enable_ip+0x90/0xd0
[ 297.313012] [<c1042880>] ? local_bh_enable_ip+0x90/0xd0
[ 297.313012] [<c16c91d0>] ? inet_putpeer+0x40/0x50
[ 297.313012] [<c103bfb5>] warn_slowpath_null+0x15/0x20
[ 297.313012] [<c1042880>] local_bh_enable_ip+0x90/0xd0
[ 297.313012] [<c17a787f>] _raw_spin_unlock_bh+0x2f/0x40
[ 297.313012] [<c16c91d0>] inet_putpeer+0x40/0x50
[ 297.313012] [<c16c581e>] ipv4_dst_destroy+0x2e/0x70
[ 297.313012] [<c1690eff>] dst_destroy+0x3f/0xd0
[ 297.313012] [<c16c587f>] dst_rcu_free+0x1f/0x30
[ 297.313012] [<c10898b0>] rcu_do_batch+0xa0/0x200
[ 297.313012] [<c1089a62>] __rcu_process_callbacks+0x52/0xb0
[ 297.313012] [<c108a1b1>] rcu_needs_cpu+0x111/0x1a0
[ 297.313012] [<c1064172>] tick_nohz_stop_sched_tick+0x152/0x5d0
[ 297.313012] [<c17a4f34>] ? schedule+0x454/0x720
[ 297.313012] [<c1001d30>] cpu_idle+0x70/0x110
[ 297.313012] [<c17a20a6>] start_secondary+0xcb/0xce
[ 297.313012] ---[ end trace aeda9d92a5036c87 ]---
[ 297.436645]
[ 297.436646] =========================================================
[ 297.437613] [ INFO: possible irq lock inversion dependency detected ]
[ 297.437613] 2.6.33-tip-00954-g8f6d799-dirty #18817
[ 297.437613] ---------------------------------------------------------
[ 297.437613] swapper/0 just changed the state of lock:
[ 297.437613] (&(&base->lock)->rlock){+.-.-.}, at: [<c1047498>] get_next_timer_interrupt+0x28/0x250
[ 297.437613] but this lock was taken by another, HARDIRQ-safe lock in the past:
[ 297.437613] (&(&np->lock)->rlock){-.-...}
[ 297.437613]
[ 297.437613] and interrupts could create inverse lock ordering between them.
[ 297.437613]
[ 297.437613]
[ 297.437613] other info that might help us debug this:
[ 297.437613] no locks held by swapper/0.
[ 297.437613]
[ 297.437613] the shortest dependencies between 2nd lock and 1st lock:
[ 297.437613] -> (&(&np->lock)->rlock){-.-...} ops: 0 {
[ 297.437613] IN-HARDIRQ-W at:
[ 297.437613] [<c1068e55>] mark_irqflags+0x155/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a7100>] _raw_spin_lock+0x40/0x70
[ 297.437613] [<c153a977>] nv_nic_irq_optimized+0x77/0x2b0
[ 297.437613] [<c1084fce>] handle_IRQ_event+0x2e/0xc0
[ 297.437613] [<c10872d3>] handle_fasteoi_irq+0x63/0xd0
[ 297.437613] IN-SOFTIRQ-W at:
[ 297.437613] [<c1068de6>] mark_irqflags+0xe6/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a728f>] _raw_spin_lock_irqsave+0x4f/0x90
[ 297.437613] [<c153ad10>] nv_start_xmit_optimized+0x90/0x570
[ 297.437613] [<c168d9ed>] dev_hard_start_xmit+0x1bd/0x240
[ 297.437613] [<c16a552c>] sch_direct_xmit+0xbc/0x180
[ 297.437613] [<c168ddd8>] dev_queue_xmit+0x368/0x4b0
[ 297.437613] [<c16ef8d7>] arp_xmit+0x57/0x60
[ 297.437613] [<c16efd45>] arp_send+0x55/0x60
[ 297.437613] [<c16f0923>] arp_solicit+0x163/0x2a0
[ 297.437613] [<c169450a>] neigh_timer_handler+0xca/0x2f0
[ 297.437613] [<c1047b68>] run_timer_softirq+0x168/0x260
[ 297.437613] [<c1041e1e>] __do_softirq+0x9e/0x140
[ 297.437613] INITIAL USE at:
[ 297.437613] [<c106a0c9>] __lock_acquire+0x1c9/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a7206>] _raw_spin_lock_irq+0x46/0x80
[ 297.437613] [<c1535098>] spin_lock_irq+0x8/0x10
[ 297.437613] [<c1792761>] nv_probe+0xa48/0xeae
[ 297.437613] [<c130059e>] local_pci_probe+0xe/0x10
[ 297.437613] [<c13012b0>] pci_device_probe+0x60/0x80
[ 297.437613] [<c13f0a19>] really_probe+0x59/0x1e0
[ 297.437613] [<c13f0bcd>] driver_probe_device+0x2d/0x80
[ 297.437613] [<c13f0c99>] __driver_attach+0x79/0x80
[ 297.437613] [<c13f01b8>] bus_for_each_dev+0x48/0x70
[ 297.437613] [<c13f0839>] driver_attach+0x19/0x20
[ 297.437613] [<c13efac7>] bus_add_driver+0x1b7/0x2b0
[ 297.437613] [<c13f0f15>] driver_register+0x65/0x120
[ 297.437613] [<c1300d84>] __pci_register_driver+0x54/0xc0
[ 297.437613] [<c1b46923>] init_nic+0x14/0x16
[ 297.437613] [<c100112d>] do_one_initcall+0x2d/0x190
[ 297.437613] [<c1b1635c>] do_basic_setup+0x42/0x52
[ 297.437613] [<c1b163de>] kernel_init+0x72/0xb9
[ 297.437613] [<c1003402>] kernel_thread_helper+0x6/0x34
[ 297.437613] }
[ 297.437613] ... key at: [<c2407564>] __key.40066+0x0/0x8
[ 297.437613] ... acquired at:
[ 297.437613] [<c106958f>] check_prev_add+0x23f/0x4d0
[ 297.437613] [<c1069dc0>] validate_chain+0x5a0/0x6e0
[ 297.437613] [<c106a1f6>] __lock_acquire+0x2f6/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a728f>] _raw_spin_lock_irqsave+0x4f/0x90
[ 297.437613] [<c10472c7>] lock_timer_base+0x27/0x50
[ 297.437613] [<c10480f6>] mod_timer+0x46/0x180
[ 297.437613] [<c153a45e>] nv_open+0x3de/0x4f0
[ 297.437613] [<c168bd1d>] dev_open+0x8d/0xe0
[ 297.437613] [<c168bb76>] dev_change_flags+0x76/0x190
[ 297.437613] [<c16f3f1c>] devinet_ioctl+0x57c/0x610
[ 297.437613] [<c16f522a>] inet_ioctl+0x9a/0xc0
[ 297.437613] [<c167b373>] sock_ioctl+0x63/0x260
[ 297.437613] [<c10cdf0d>] vfs_ioctl+0x2d/0xc0
[ 297.437613] [<c10ce7b7>] do_vfs_ioctl+0x67/0x320
[ 297.437613] [<c10ceacf>] sys_ioctl+0x5f/0x80
[ 297.437613] [<c1002e73>] sysenter_do_call+0x12/0x32
[ 297.437613]
[ 297.437613] -> (&(&base->lock)->rlock){+.-.-.} ops: 0 {
[ 297.437613] HARDIRQ-ON-W at:
[ 297.437613] [<c1068e06>] mark_irqflags+0x106/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a7100>] _raw_spin_lock+0x40/0x70
[ 297.437613] [<c1047498>] get_next_timer_interrupt+0x28/0x250
[ 297.437613] [<c1064328>] tick_nohz_stop_sched_tick+0x308/0x5d0
[ 297.437613] [<c1001d30>] cpu_idle+0x70/0x110
[ 297.437613] [<c17a20a6>] start_secondary+0xcb/0xce
[ 297.437613] IN-SOFTIRQ-W at:
[ 297.437613] [<c1068de6>] mark_irqflags+0xe6/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a7206>] _raw_spin_lock_irq+0x46/0x80
[ 297.437613] [<c1047a37>] run_timer_softirq+0x37/0x260
[ 297.437613] [<c1041e1e>] __do_softirq+0x9e/0x140
[ 297.437613] IN-RECLAIM_FS-W at:
[ 297.437613] [<c1068db2>] mark_irqflags+0xb2/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a728f>] _raw_spin_lock_irqsave+0x4f/0x90
[ 297.437613] [<c10472c7>] lock_timer_base+0x27/0x50
[ 297.437613] [<c17a54aa>] schedule_timeout+0x7a/0x210
[ 297.437613] [<c10a48c7>] kswapd+0x1b7/0x1e0
[ 297.437613] [<c1052e2c>] kthread+0x7c/0x90
[ 297.437613] [<c1003402>] kernel_thread_helper+0x6/0x34
[ 297.437613] INITIAL USE at:
[ 297.437613] [<c106a0c9>] __lock_acquire+0x1c9/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a728f>] _raw_spin_lock_irqsave+0x4f/0x90
[ 297.437613] [<c10472c7>] lock_timer_base+0x27/0x50
[ 297.437613] [<c10480f6>] mod_timer+0x46/0x180
[ 297.437613] [<c1b36c85>] con_init+0xae/0x1fb
[ 297.437613] [<c1b364a1>] console_init+0x12/0x20
[ 297.437613] [<c1b1681d>] start_kernel+0x222/0x2ed
[ 297.437613] [<c1b16089>] i386_start_kernel+0x89/0x8b
[ 297.437613] }
[ 297.437613] ... key at: [<c1e071c0>] __key.30254+0x0/0x8
[ 297.437613] ... acquired at:
[ 297.437613] [<c1068402>] check_usage_backwards+0x82/0xe0
[ 297.437613] [<c10689b4>] mark_lock_irq+0x84/0x240
[ 297.437613] [<c1068c72>] mark_lock+0x102/0x190
[ 297.437613] [<c1068e06>] mark_irqflags+0x106/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c17a7100>] _raw_spin_lock+0x40/0x70
[ 297.437613] [<c1047498>] get_next_timer_interrupt+0x28/0x250
[ 297.437613] [<c1064328>] tick_nohz_stop_sched_tick+0x308/0x5d0
[ 297.437613] [<c1001d30>] cpu_idle+0x70/0x110
[ 297.437613] [<c17a20a6>] start_secondary+0xcb/0xce
[ 297.437613]
[ 297.437613]
[ 297.437613] stack backtrace:
[ 297.437613] Pid: 0, comm: swapper Tainted: G W 2.6.33-tip-00954-g8f6d799-dirty #18817
[ 297.437613] Call Trace:
[ 297.437613] [<c17a46c1>] ? printk+0x18/0x1a
[ 297.437613] [<c1067f64>] print_irq_inversion_bug+0x104/0x120
[ 297.437613] [<c1068402>] check_usage_backwards+0x82/0xe0
[ 297.437613] [<c1005daf>] ? dump_trace+0x7f/0xd0
[ 297.437613] [<c10689b4>] mark_lock_irq+0x84/0x240
[ 297.437613] [<c1068380>] ? check_usage_backwards+0x0/0xe0
[ 297.437613] [<c1068c72>] mark_lock+0x102/0x190
[ 297.437613] [<c1068e06>] mark_irqflags+0x106/0x160
[ 297.437613] [<c106a29a>] __lock_acquire+0x39a/0x470
[ 297.437613] [<c106919b>] ? trace_hardirqs_on+0xb/0x10
[ 297.437613] [<c10b8c76>] ? slob_free+0xa6/0x310
[ 297.437613] [<c106a3d4>] lock_acquire+0x64/0x80
[ 297.437613] [<c1047498>] ? get_next_timer_interrupt+0x28/0x250
[ 297.437613] [<c17a7100>] _raw_spin_lock+0x40/0x70
[ 297.437613] [<c1047498>] ? get_next_timer_interrupt+0x28/0x250
[ 297.437613] [<c1047498>] get_next_timer_interrupt+0x28/0x250
[ 297.437613] [<c1089a62>] ? __rcu_process_callbacks+0x52/0xb0
[ 297.437613] [<c108a1b1>] ? rcu_needs_cpu+0x111/0x1a0
[ 297.437613] [<c1064328>] tick_nohz_stop_sched_tick+0x308/0x5d0
[ 297.437613] [<c17a4f34>] ? schedule+0x454/0x720
[ 297.437613] [<c1001d30>] cpu_idle+0x70/0x110
[ 297.437613] [<c17a20a6>] start_secondary+0xcb/0xce
Ingo
View attachment "config" of type "text/plain" (73970 bytes)
Powered by blists - more mailing lists