| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Sun, 15 Dec 2013 14:39:25 +0800
From: Fengguang Wu <fengguang.wu@...el.com>
To: Eric Dumazet <edumazet@...gle.com>
Cc: "David S. Miller" <davem@...emloft.net>, netdev@...r.kernel.org,
linux-kernel@...r.kernel.org
Subject: __udp4_lib_rcv: inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
Greetings,
I got the below dmesg and the first bad commit is
commit 975022310233fb0f0193873d79a7b8438070fa82
Author: Eric Dumazet <edumazet@...gle.com>
AuthorDate: Wed Dec 11 14:46:51 2013 -0800
Commit: David S. Miller <davem@...emloft.net>
CommitDate: Wed Dec 11 20:21:10 2013 -0500
udp: ipv4: must add synchronization in udp_sk_rx_dst_set()
Unlike TCP, UDP input path does not hold the socket lock.
Before messing with sk->sk_rx_dst, we must use a spinlock, otherwise
multiple cpus could leak a refcount.
This patch also takes care of renewing a stale dst entry.
(When the sk->sk_rx_dst would not be used by IP early demux)
Fixes: 421b3885bf6d ("udp: ipv4: Add udp early demux")
Signed-off-by: Eric Dumazet <edumazet@...gle.com>
Cc: Shawn Bohrer <sbohrer@...advisors.com>
Signed-off-by: David S. Miller <davem@...emloft.net>
+----------------------------------------------+----+
| | |
+----------------------------------------------+----+
| boot_successes | 0 |
| boot_failures | 19 |
| inconsistent_SOFTIRQ-ON-W-IN-SOFTIRQ-W_usage | 19 |
+----------------------------------------------+----+
[ 23.564204] [ INFO: inconsistent lock state ]
[ 23.564204] 3.13.0-rc3-00689-gb1daf37 #892 Not tainted
[ 23.564204] ---------------------------------
[ 23.564204] inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
[ 23.564204] swapper/0/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
[ 23.564204] (&(&sk->sk_dst_lock)->rlock){+.?...}, at: [<ffffffff81a27fe2>] __udp4_lib_rcv+0x569/0x79d
[ 23.564204] {SOFTIRQ-ON-W} state was registered at:
[ 23.564204] [<ffffffff810b16b7>] __lock_acquire+0x96c/0x185f
[ 23.564204] [<ffffffff810b2c97>] lock_acquire+0xab/0x12e
[ 23.564204] [<ffffffff81af12d3>] _raw_spin_lock+0x3b/0x6d
[ 23.564204] [<ffffffff81a2f13d>] inet_bind+0x18d/0x1dc
[ 23.564204] [<ffffffff8199d62c>] kernel_bind+0x10/0x12
[ 23.564204] [<ffffffff81aae764>] xs_bind+0xa0/0x11e
[ 23.564204] [<ffffffff81ab0271>] xs_create_sock.isra.14+0x20b/0x230
[ 23.564204] [<ffffffff81ab047f>] xs_tcp_setup_socket+0x5f/0x3e5
[ 23.564204] [<ffffffff8108a0d7>] process_one_work+0x247/0x443
[ 23.564204] [<ffffffff8108aeac>] worker_thread+0x1d2/0x2bc
[ 23.564204] [<ffffffff81090228>] kthread+0xf9/0x101
[ 23.564204] [<ffffffff81af92fc>] ret_from_fork+0x7c/0xb0
[ 23.564204] irq event stamp: 514170
[ 23.564204] hardirqs last enabled at (514170): [<ffffffff810766af>] local_bh_enable+0xab/0xb5
[ 23.564204] hardirqs last disabled at (514169): [<ffffffff8107665a>] local_bh_enable+0x56/0xb5
[ 23.564204] softirqs last enabled at (514148): [<ffffffff81076106>] _local_bh_enable+0x43/0x45
[ 23.564204] softirqs last disabled at (514149): [<ffffffff81076779>] irq_exit+0x59/0xa0
[ 23.564204]
[ 23.564204] other info that might help us debug this:
[ 23.564204] Possible unsafe locking scenario:
[ 23.564204]
[ 23.564204] CPU0
[ 23.564204] ----
[ 23.564204] lock(&(&sk->sk_dst_lock)->rlock);
[ 23.564204] <Interrupt>
[ 23.564204] lock(&(&sk->sk_dst_lock)->rlock);
[ 23.564204]
[ 23.564204] *** DEADLOCK ***
[ 23.564204]
[ 23.564204] 2 locks held by swapper/0/0:
[ 23.564204] #0: (rcu_read_lock){.+.+..}, at: [<ffffffff819b258a>] __netif_receive_skb_core+0x168/0x58e
[ 23.564204] #1: (rcu_read_lock){.+.+..}, at: [<ffffffff81a0086a>] ip_local_deliver_finish+0x2f/0x2a0
[ 23.564204]
[ 23.564204] stack backtrace:
[ 23.564204] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.13.0-rc3-00689-gb1daf37 #892
[ 23.564204] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 23.564204] 0000000000000002 ffff88001ea039a0 ffffffff81ae56fd ffffffff82097490
[ 23.564204] ffff88001ea039f0 ffffffff81addae7 0000000000000001 ffff880000000001
[ 23.564204] ffff880000000000 ffffffff82097c08 0000000000000004 0000000000000006
[ 23.564204] Call Trace:
[ 23.564204] <IRQ> [<ffffffff81ae56fd>] dump_stack+0x4d/0x66
[ 23.564204] [<ffffffff81addae7>] print_usage_bug+0x2a8/0x2b9
[ 23.564204] [<ffffffff810af93a>] ? print_shortest_lock_dependencies+0x193/0x193
[ 23.564204] [<ffffffff810b05b5>] mark_lock+0x33c/0x57c
[ 23.564204] [<ffffffff810b1645>] __lock_acquire+0x8fa/0x185f
[ 23.564204] [<ffffffff8109fd9c>] ? sched_clock_local+0x12/0x72
[ 23.564204] [<ffffffff810addfa>] ? trace_hardirqs_off_caller+0x1f/0xb2
[ 23.564204] [<ffffffff810b2c97>] lock_acquire+0xab/0x12e
[ 23.564204] [<ffffffff81a27fe2>] ? __udp4_lib_rcv+0x569/0x79d
[ 23.564204] [<ffffffff810addfa>] ? trace_hardirqs_off_caller+0x1f/0xb2
[ 23.564204] [<ffffffff81af12d3>] _raw_spin_lock+0x3b/0x6d
[ 23.564204] [<ffffffff81a27fe2>] ? __udp4_lib_rcv+0x569/0x79d
[ 23.564204] [<ffffffff81a27fe2>] __udp4_lib_rcv+0x569/0x79d
[ 23.564204] [<ffffffff81a286ae>] udp_rcv+0x1a/0x1c
[ 23.564204] [<ffffffff81a0098c>] ip_local_deliver_finish+0x151/0x2a0
[ 23.564204] [<ffffffff81a0086a>] ? ip_local_deliver_finish+0x2f/0x2a0
[ 23.564204] [<ffffffff81a0109e>] ip_local_deliver+0x7a/0x7f
[ 23.564204] [<ffffffff81a00e75>] ip_rcv_finish+0x39a/0x461
[ 23.564204] [<ffffffff81a01326>] ip_rcv+0x283/0x2e0
[ 23.564204] [<ffffffff819b28ef>] __netif_receive_skb_core+0x4cd/0x58e
[ 23.564204] [<ffffffff819b258a>] ? __netif_receive_skb_core+0x168/0x58e
[ 23.564204] [<ffffffff819b2dd6>] __netif_receive_skb+0x1d/0x5f
[ 23.564204] [<ffffffff819b2e80>] netif_receive_skb+0x68/0xf4
[ 23.564204] [<ffffffff819b35da>] napi_gro_receive+0x3e/0x80
[ 23.564204] [<ffffffff817be870>] e1000_clean_rx_irq+0x393/0x415
[ 23.564204] [<ffffffff817bcd48>] e1000_clean+0x59d/0x71f
[ 23.564204] [<ffffffff810634d1>] ? kvm_clock_read+0x27/0x31
[ 23.564204] [<ffffffff8103edd0>] ? sched_clock+0x9/0xd
[ 23.564204] [<ffffffff819b3ed0>] net_rx_action+0xf6/0x210
[ 23.564204] [<ffffffff810762fc>] __do_softirq+0xf3/0x27a
[ 23.564204] [<ffffffff81076779>] irq_exit+0x59/0xa0
[ 23.564204] [<ffffffff81afb44d>] do_IRQ+0x9d/0xb4
[ 23.564204] [<ffffffff81af21f2>] common_interrupt+0x72/0x72
[ 23.564204] <EOI> [<ffffffff81af5ab5>] ? __atomic_notifier_call_chain+0x5/0x9c
[ 23.564204] [<ffffffff810636c8>] ? native_safe_halt+0x6/0x8
[ 23.564204] [<ffffffff8103f9d1>] default_idle+0x38/0xc1
[ 23.564204] [<ffffffff81040154>] arch_cpu_idle+0x18/0x28
[ 23.564204] [<ffffffff810c0ac5>] cpu_startup_entry+0x15c/0x24d
[ 23.564204] [<ffffffff81ad1f25>] rest_init+0xc9/0xcd
[ 23.564204] [<ffffffff81ad1e61>] ? rest_init+0x5/0xcd
[ 23.564204] [<ffffffff823aed77>] start_kernel+0x3d4/0x3e0
[ 23.564204] [<ffffffff823ae7b1>] ? repair_env_string+0x58/0x58
[ 23.564204] [<ffffffff823ae120>] ? early_idt_handlers+0x120/0x120
[ 23.564204] [<ffffffff823ae467>] x86_64_start_reservations+0x2a/0x2c
[ 23.564204] [<ffffffff823ae5a9>] x86_64_start_kernel+0x140/0x14d
[ 66.435785] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[ 66.440815] NFSD: starting 90-second grace period (net ffffffff82196a80)
Kernel tests: Boot OK!
git bisect start b1daf37ff52cf2ff39b2db2399e052ad7965c6aa 374b105797c3d4f29c685f3be535c35f5689b30e --
git bisect bad c2f84f5177293270c0203fabfd600e11e211fd73 # 12:07 0- 1 Merge 'net/master' into devel-xian-x86_64-201312131052
git bisect good 2481b0b9a5f7af0f748a18a9710371a9cb22c986 # 12:41 20+ 1 Merge 'arm-soc/to-build' into devel-xian-x86_64-201312131052
git bisect good c8781cf4a309ae4d1393f5878d4e51987665898c # 13:08 20+ 0 bnx2x: avoid null pointer dereference when enabling SR-IOV
git bisect good fffc15a5012e9052d3b236efc56840841a125416 # 13:36 20+ 0 vxlan: release rt when found circular route
git bisect good 4bd7b5127bd02c12c1cc837a7a0b6ce295eb2505 # 13:50 20+ 0 micrel: add support for KSZ8041RNLI
git bisect good 9508fdde4d53f0d9e583e841ed08796d2f310be1 # 14:14 20+ 1 Revert "8390 : Replace ei_debug with msg_enable/NETIF_MSG_* feature"
git bisect bad 10574059ce0451c6572c85329c772aa15085f8eb # 14:28 0- 1 xen-netback: napi: fix abuse of budget
git bisect bad 975022310233fb0f0193873d79a7b8438070fa82 # 14:28 0- 8 udp: ipv4: must add synchronization in udp_sk_rx_dst_set()
git bisect good a1bf1750871a6f242b0fdb174cc55d2c57e7ed66 # 14:34 25+ 0 net:fec: remove duplicate lines in comment about errata ERR006358
# first bad commit: [975022310233fb0f0193873d79a7b8438070fa82] udp: ipv4: must add synchronization in udp_sk_rx_dst_set()
git bisect good a1bf1750871a6f242b0fdb174cc55d2c57e7ed66 # 14:46 75+ 0 net:fec: remove duplicate lines in comment about errata ERR006358
git bisect bad b1daf37ff52cf2ff39b2db2399e052ad7965c6aa # 14:46 0- 19 Merge 'security/for-linus' into devel-xian-x86_64-201312131052
git bisect good 2cd6a37a65f578f7e23e67986107422ad3c012ba # 15:23 75+ 0 Revert "udp: ipv4: must add synchronization in udp_sk_rx_dst_set()"
git bisect good 8ae516aa8b8161254d3e402b3348b2a9b8d1efd0 # 15:54 75+ 1 Merge tag 'trace-fixes-v3.13-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace
git bisect bad 1a63d5777d237952bb06f1a2481c3acc366aeb8d # 15:57 0- 9 Add linux-next specific files for 20131213
Thanks,
Fengguang
View attachment "dmesg-nfsroot-cairo-7:20131213110650:x86_64-nfsroot:3.13.0-rc3-00689-gb1daf37:892" of type "text/plain" (77308 bytes)
Download attachment "x86_64-nfsroot-b1daf37ff52cf2ff39b2db2399e052ad7965c6aa-inconsistent--130714.log" of type "application/octet-stream" (36484 bytes)
View attachment "config-3.13.0-rc3-00689-gb1daf37" of type "text/plain" (97106 bytes)
_______________________________________________
LKP mailing list
LKP@...ux.intel.com
Powered by blists - more mailing lists