lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 3 Nov 2016 18:21:24 +0100
From:   Andrey Konovalov <andreyknvl@...gle.com>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     Herbert Xu <herbert@...dor.apana.org.au>,
        Andrew Morton <akpm@...ux-foundation.org>,
        David Decotigny <decot@...glers.com>,
        "David S. Miller" <davem@...emloft.net>,
        Dmitry Ivanov <dmitrijs.ivanovs@...t.com>,
        Eric Dumazet <edumazet@...gle.com>,
        Florian Westphal <fw@...len.de>,
        Greg Rose <grose@...htfleet.com>,
        Johannes Berg <johannes.berg@...el.com>,
        Matti Vaittinen <matti.vaittinen@...ia.com>,
        Pravin B Shelar <pshelar@....org>,
        stephen hemminger <stephen@...workplumber.org>,
        Tom Herbert <tom@...bertland.com>,
        Tycho Andersen <tycho.andersen@...onical.com>,
        LKML <linux-kernel@...r.kernel.org>,
        netdev <netdev@...r.kernel.org>,
        syzkaller <syzkaller@...glegroups.com>,
        Kostya Serebryany <kcc@...gle.com>,
        Alexander Potapenko <glider@...gle.com>,
        Dmitry Vyukov <dvyukov@...gle.com>
Subject: Re: net/netlink: null-ptr-deref in netlink_dump/lock_acquire

Hi,

Another report that looks related:

[ INFO: possible circular locking dependency detected ]
4.9.0-rc3+ #344 Not tainted
-------------------------------------------------------
syz-executor/25526 is trying to acquire lock:
 ([  950.351060] &table[i].mutex
[<ffffffff82dcf798>] nfnl_lock+0x28/0x30 net/netfilter/nfnetlink.c:61

but task is already holding lock:
 ([  950.351060] rtnl_mutex
[<     inline     >] rtnl_lock net/core/rtnetlink.c:70
[<ffffffff82c15efb>] rtnetlink_rcv+0x1b/0x40 net/core/rtnetlink.c:4032

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

:
       [  950.351060] [<ffffffff8120c52e>] lock_acquire+0x17e/0x340
kernel/locking/lockdep.c:3746
       [  950.351060] [<     inline     >] __mutex_lock_common
kernel/locking/mutex.c:521
       [  950.351060] [<ffffffff83fb7221>]
mutex_lock_nested+0xb1/0x890 kernel/locking/mutex.c:621
       [  950.351060] [<ffffffff82c11297>] rtnl_lock+0x17/0x20
net/core/rtnetlink.c:70
       [  950.351060] [<ffffffff839fe31f>]
nl80211_get_reg_dump+0x5f/0x400 net/wireless/nl80211.c:6171
       [  950.351060] [<ffffffff82dc18d8>] genl_lock_dumpit+0x68/0x90
net/netlink/genetlink.c:517
       [  950.351060] [<ffffffff82db6947>] netlink_dump+0x397/0xac0
net/netlink/af_netlink.c:2110
       [  950.351060] [<ffffffff82db99b1>]
__netlink_dump_start+0x501/0x770 net/netlink/af_netlink.c:2200
       [  950.351060] [<ffffffff82dc2cb8>]
genl_family_rcv_msg+0xad8/0xc80 net/netlink/genetlink.c:584
       [  950.351060] [<ffffffff82dc3016>] genl_rcv_msg+0x1b6/0x270
net/netlink/genetlink.c:658
       [  950.351060] [<ffffffff82dc10a0>] netlink_rcv_skb+0x2c0/0x3b0
net/netlink/af_netlink.c:2281
       [  950.351060] [<ffffffff82dc21c8>] genl_rcv+0x28/0x40
net/netlink/genetlink.c:669
       [  950.351060] [<     inline     >] netlink_unicast_kernel
net/netlink/af_netlink.c:1214
       [  950.351060] [<ffffffff82dbf959>] netlink_unicast+0x5a9/0x880
net/netlink/af_netlink.c:1240
       [  950.351060] [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0
net/netlink/af_netlink.c:1786
       [  950.351060] [<     inline     >] sock_sendmsg_nosec net/socket.c:606
       [  950.351060] [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110
net/socket.c:616
       [  950.351060] [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0
net/socket.c:814
       [  950.351060] [<     inline     >] new_sync_write fs/read_write.c:499
       [  950.351060] [<ffffffff8151bd44>] __vfs_write+0x334/0x570
fs/read_write.c:512
       [  950.351060] [<ffffffff8151f85b>] vfs_write+0x17b/0x500
fs/read_write.c:560
       [  950.351060] [<     inline     >] SYSC_write fs/read_write.c:607
       [  950.351060] [<ffffffff81523184>] SyS_write+0xd4/0x1a0
fs/read_write.c:599
       [  950.351060] [<ffffffff83fc0281>] entry_SYSCALL_64_fastpath+0x1f/0xc2

:
       [  950.351060] [<ffffffff8120c52e>] lock_acquire+0x17e/0x340
kernel/locking/lockdep.c:3746
       [  950.351060] [<     inline     >] __mutex_lock_common
kernel/locking/mutex.c:521
       [  950.351060] [<ffffffff83fb7221>]
mutex_lock_nested+0xb1/0x890 kernel/locking/mutex.c:621
       [  950.351060] [<     inline     >] genl_lock net/netlink/genetlink.c:31
       [  950.351060] [<ffffffff82dc18b1>] genl_lock_dumpit+0x41/0x90
net/netlink/genetlink.c:516
       [  950.351060] [<ffffffff82db6947>] netlink_dump+0x397/0xac0
net/netlink/af_netlink.c:2110
       [  950.351060] [<ffffffff82db99b1>]
__netlink_dump_start+0x501/0x770 net/netlink/af_netlink.c:2200
       [  950.351060] [<ffffffff82dc2cb8>]
genl_family_rcv_msg+0xad8/0xc80 net/netlink/genetlink.c:584
       [  950.351060] [<ffffffff82dc3016>] genl_rcv_msg+0x1b6/0x270
net/netlink/genetlink.c:658
       [  950.351060] [<ffffffff82dc10a0>] netlink_rcv_skb+0x2c0/0x3b0
net/netlink/af_netlink.c:2281
       [  950.351060] [<ffffffff82dc21c8>] genl_rcv+0x28/0x40
net/netlink/genetlink.c:669
       [  950.351060] [<     inline     >] netlink_unicast_kernel
net/netlink/af_netlink.c:1214
       [  950.351060] [<ffffffff82dbf959>] netlink_unicast+0x5a9/0x880
net/netlink/af_netlink.c:1240
       [  950.351060] [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0
net/netlink/af_netlink.c:1786
       [  950.351060] [<     inline     >] sock_sendmsg_nosec net/socket.c:606
       [  950.351060] [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110
net/socket.c:616
       [  950.351060] [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0
net/socket.c:814
       [  950.351060] [<     inline     >] new_sync_write fs/read_write.c:499
       [  950.351060] [<ffffffff8151bd44>] __vfs_write+0x334/0x570
fs/read_write.c:512
       [  950.351060] [<ffffffff8151f85b>] vfs_write+0x17b/0x500
fs/read_write.c:560
       [  950.351060] [<     inline     >] SYSC_write fs/read_write.c:607
       [  950.351060] [<ffffffff81523184>] SyS_write+0xd4/0x1a0
fs/read_write.c:599
       [  950.351060] [<ffffffff83fc0281>] entry_SYSCALL_64_fastpath+0x1f/0xc2

:
       [  950.351060] [<ffffffff8120c52e>] lock_acquire+0x17e/0x340
kernel/locking/lockdep.c:3746
       [  950.351060] [<     inline     >] __mutex_lock_common
kernel/locking/mutex.c:521
       [  950.351060] [<ffffffff83fb7221>]
mutex_lock_nested+0xb1/0x890 kernel/locking/mutex.c:621
       [  950.351060] [<ffffffff82db95aa>]
__netlink_dump_start+0xfa/0x770 net/netlink/af_netlink.c:2170
       [  950.351060] [<     inline     >] netlink_dump_start
include/linux/netlink.h:165
       [  950.351060] [<ffffffff82e19309>]
ctnetlink_stat_ct_cpu+0xd9/0x130
net/netfilter/nf_conntrack_netlink.c:2045
       [  950.351060] [<ffffffff82dd1d80>]
nfnetlink_rcv_msg+0xa10/0xc10 net/netfilter/nfnetlink.c:212
       [  950.351060] [<ffffffff82dc10a0>] netlink_rcv_skb+0x2c0/0x3b0
net/netlink/af_netlink.c:2281
       [  950.351060] [<ffffffff82dd07e8>] nfnetlink_rcv+0x848/0x1170
net/netfilter/nfnetlink.c:474
       [  950.351060] [<     inline     >] netlink_unicast_kernel
net/netlink/af_netlink.c:1214
       [  950.351060] [<ffffffff82dbf959>] netlink_unicast+0x5a9/0x880
net/netlink/af_netlink.c:1240
       [  950.351060] [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0
net/netlink/af_netlink.c:1786
       [  950.351060] [<     inline     >] sock_sendmsg_nosec net/socket.c:606
       [  950.351060] [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110
net/socket.c:616
       [  950.351060] [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0
net/socket.c:814
       [  950.351060] [<     inline     >] new_sync_write fs/read_write.c:499
       [  950.351060] [<ffffffff8151bd44>] __vfs_write+0x334/0x570
fs/read_write.c:512
       [  950.351060] [<ffffffff8151f85b>] vfs_write+0x17b/0x500
fs/read_write.c:560
       [  950.351060] [<     inline     >] SYSC_write fs/read_write.c:607
       [  950.351060] [<ffffffff81523184>] SyS_write+0xd4/0x1a0
fs/read_write.c:599
       [  950.351060] [<ffffffff83fc0281>] entry_SYSCALL_64_fastpath+0x1f/0xc2

:
       [  950.351060] [<     inline     >] check_prev_add
kernel/locking/lockdep.c:1829
       [  950.351060] [<     inline     >] check_prevs_add
kernel/locking/lockdep.c:1939
       [  950.351060] [<     inline     >] validate_chain
kernel/locking/lockdep.c:2266
       [  950.351060] [<ffffffff8120b0ca>]
__lock_acquire+0x2b4a/0x3450 kernel/locking/lockdep.c:3335
       [  950.351060] [<ffffffff8120c52e>] lock_acquire+0x17e/0x340
kernel/locking/lockdep.c:3746
       [  950.351060] [<     inline     >] __mutex_lock_common
kernel/locking/mutex.c:521
       [  950.351060] [<ffffffff83fb7221>]
mutex_lock_nested+0xb1/0x890 kernel/locking/mutex.c:621
       [  950.351060] [<ffffffff82dcf798>] nfnl_lock+0x28/0x30
net/netfilter/nfnetlink.c:61
       [  950.351060] [<ffffffff82e847bf>]
nf_tables_netdev_event+0x14f/0x590
net/netfilter/nf_tables_netdev.c:122
       [  950.351060] [<ffffffff8117c131>]
notifier_call_chain+0x91/0x1a0 kernel/notifier.c:93
       [  950.351060] [<     inline     >] __raw_notifier_call_chain
kernel/notifier.c:394
       [  950.351060] [<ffffffff8117c2bd>]
raw_notifier_call_chain+0x2d/0x40 kernel/notifier.c:401
       [  950.351060] [<ffffffff82bc3851>]
call_netdevice_notifiers_info+0x51/0x90 net/core/dev.c:1645
       [  950.351060] [<     inline     >] call_netdevice_notifiers
net/core/dev.c:1661
       [  950.351060] [<ffffffff82be77c7>] dev_change_name+0x5d7/0x920
net/core/dev.c:1204
       [  950.351060] [<ffffffff82c171dd>] do_setlink+0x83d/0x30d0
net/core/rtnetlink.c:1993
       [  950.351060] [<ffffffff82c19ce0>] rtnl_setlink+0x270/0x330
net/core/rtnetlink.c:2231
       [  950.351060] [<ffffffff82c254a4>]
rtnetlink_rcv_msg+0x274/0x700 net/core/rtnetlink.c:4027
       [  950.351060] [<ffffffff82dc10a0>] netlink_rcv_skb+0x2c0/0x3b0
net/netlink/af_netlink.c:2281
       [  950.351060] [<ffffffff82c15f0a>] rtnetlink_rcv+0x2a/0x40
net/core/rtnetlink.c:4033
       [  950.351060] [<     inline     >] netlink_unicast_kernel
net/netlink/af_netlink.c:1214
       [  950.351060] [<ffffffff82dbf959>] netlink_unicast+0x5a9/0x880
net/netlink/af_netlink.c:1240
       [  950.351060] [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0
net/netlink/af_netlink.c:1786
       [  950.351060] [<     inline     >] sock_sendmsg_nosec net/socket.c:606
       [  950.351060] [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110
net/socket.c:616
       [  950.351060] [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0
net/socket.c:814
       [  950.351060] [<     inline     >] new_sync_write fs/read_write.c:499
       [  950.351060] [<ffffffff8151bd44>] __vfs_write+0x334/0x570
fs/read_write.c:512
       [  950.351060] [<ffffffff8151f85b>] vfs_write+0x17b/0x500
fs/read_write.c:560
       [  950.351060] [<     inline     >] SYSC_write fs/read_write.c:607
       [  950.351060] [<ffffffff81523184>] SyS_write+0xd4/0x1a0
fs/read_write.c:599
       [  950.351060] [<ffffffff81006465>] do_syscall_64+0x195/0x490
arch/x86/entry/common.c:280
       [  950.351060] [<ffffffff83fc0349>] return_from_SYSCALL_64+0x0/0x7a

other info that might help us debug this:

Chain exists of:


 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock([  950.351060] rtnl_mutex
);
                               lock([  950.351060] genl_mutex
);
                               lock([  950.351060] rtnl_mutex
);
  lock([  950.351060] &table[i].mutex
);

 *** DEADLOCK ***

1 lock held by syz-executor/25526:
 #0: [  950.351060]  (

stack backtrace:
CPU: 0 PID: 25526 Comm: syz-executor Not tainted 4.9.0-rc3+ #344
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
 ffff8800692b6d30 ffffffff81b46934 ffffffff859bdf80 ffffffff859de740
 ffffffff859b3870 1ffffffff0bade82 ffff8800692b6d80 ffffffff81204cfd
 ffffffff859b3870 ffff88006859cca0 000000006859c480 ffff88006859cca0
Call Trace:
 [<     inline     >] __dump_stack lib/dump_stack.c:15
 [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
 [<ffffffff81204cfd>] print_circular_bug+0x28d/0x350
kernel/locking/lockdep.c:1202
 [<     inline     >] check_prev_add kernel/locking/lockdep.c:1829
 [<     inline     >] check_prevs_add kernel/locking/lockdep.c:1939
 [<     inline     >] validate_chain kernel/locking/lockdep.c:2266
 [<ffffffff8120b0ca>] __lock_acquire+0x2b4a/0x3450 kernel/locking/lockdep.c:3335
 [<ffffffff8120c52e>] lock_acquire+0x17e/0x340 kernel/locking/lockdep.c:3746
 [<     inline     >] __mutex_lock_common kernel/locking/mutex.c:521
 [<ffffffff83fb7221>] mutex_lock_nested+0xb1/0x890 kernel/locking/mutex.c:621
 [<ffffffff82dcf798>] nfnl_lock+0x28/0x30 net/netfilter/nfnetlink.c:61
 [<ffffffff82e847bf>] nf_tables_netdev_event+0x14f/0x590
net/netfilter/nf_tables_netdev.c:122
 [<ffffffff8117c131>] notifier_call_chain+0x91/0x1a0 kernel/notifier.c:93
 [<     inline     >] __raw_notifier_call_chain kernel/notifier.c:394
 [<ffffffff8117c2bd>] raw_notifier_call_chain+0x2d/0x40 kernel/notifier.c:401
 [<ffffffff82bc3851>] call_netdevice_notifiers_info+0x51/0x90
net/core/dev.c:1645
 [<     inline     >] call_netdevice_notifiers net/core/dev.c:1661
 [<ffffffff82be77c7>] dev_change_name+0x5d7/0x920 net/core/dev.c:1204
 [<ffffffff82c171dd>] do_setlink+0x83d/0x30d0 net/core/rtnetlink.c:1993
 [<ffffffff82c19ce0>] rtnl_setlink+0x270/0x330 net/core/rtnetlink.c:2231
 [<ffffffff82c254a4>] rtnetlink_rcv_msg+0x274/0x700 net/core/rtnetlink.c:4027
 [<ffffffff82dc10a0>] netlink_rcv_skb+0x2c0/0x3b0 net/netlink/af_netlink.c:2281
 [<ffffffff82c15f0a>] rtnetlink_rcv+0x2a/0x40 net/core/rtnetlink.c:4033
 [<     inline     >] netlink_unicast_kernel net/netlink/af_netlink.c:1214
 [<ffffffff82dbf959>] netlink_unicast+0x5a9/0x880 net/netlink/af_netlink.c:1240
 [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0 net/netlink/af_netlink.c:1786
 [<     inline     >] sock_sendmsg_nosec net/socket.c:606
 [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110 net/socket.c:616
 [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0 net/socket.c:814
 [<     inline     >] new_sync_write fs/read_write.c:499
 [<ffffffff8151bd44>] __vfs_write+0x334/0x570 fs/read_write.c:512
 [<ffffffff8151f85b>] vfs_write+0x17b/0x500 fs/read_write.c:560
 [<     inline     >] SYSC_write fs/read_write.c:607
 [<ffffffff81523184>] SyS_write+0xd4/0x1a0 fs/read_write.c:599
 [<ffffffff81006465>] do_syscall_64+0x195/0x490 arch/x86/entry/common.c:280
 [<ffffffff83fc0349>] entry_SYSCALL64_slow_path+0x25/0x25

On Thu, Nov 3, 2016 at 4:08 AM, Andrey Konovalov <andreyknvl@...gle.com> wrote:
> Hi Eric,
>
> This fixes the second report, the first one is still there.
> Apparently these are two separate issues.
>
> For the second one:
> Tested-by: Andrey Konovalov <andreyknvl@...gle.com>
>
> Thanks for the fix!
>
> On Thu, Nov 3, 2016 at 3:58 AM, Eric Dumazet <eric.dumazet@...il.com> wrote:
>> On Thu, 2016-11-03 at 03:36 +0100, Andrey Konovalov wrote:
>>> On Thu, Nov 3, 2016 at 1:15 AM, Andrey Konovalov <andreyknvl@...gle.com> wrote:
>>> > On Wed, Oct 19, 2016 at 4:13 PM, Andrey Konovalov <andreyknvl@...gle.com> wrote:
>>> >> Hi,
>>> >>
>>> >> I've got the following error report while running the syzkaller fuzzer:
>>> >>
>>> >> kasan: CONFIG_KASAN_INLINE enabled
>>> >> kasan: GPF could be caused by NULL-ptr deref or user memory access
>>> >> general protection fault: 0000 [#1] SMP KASAN
>>> >> Modules linked in:
>>> >> CPU: 1 PID: 3933 Comm: syz-executor Not tainted 4.9.0-rc1+ #230
>>> >> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>> >> task: ffff88006b79d800 task.stack: ffff88006bbc0000
>>> >> RIP: 0010:[<ffffffff8120872d>]  [<ffffffff8120872d>]
>>> >> __lock_acquire+0x12d/0x3450 kernel/locking/lockdep.c:3221
>>> >> RSP: 0018:ffff88006bbc7420  EFLAGS: 00010006
>>> >> RAX: 0000000000000046 RBX: dffffc0000000000 RCX: 0000000000000000
>>> >> RDX: 000000000000000c RSI: 0000000000000000 RDI: 0000000000000003
>>> >> RBP: ffff88006bbc75c0 R08: 0000000000000001 R09: 0000000000000000
>>> >> R10: 0000000000000000 R11: ffffffff85f42240 R12: ffff88006b79d800
>>> >> R13: ffffffff84bfe4e0 R14: 0000000000000001 R15: 0000000000000060
>>> >> FS:  00007fd9c41cc700(0000) GS:ffff88006cd00000(0000) knlGS:0000000000000000
>>> >> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> >> CR2: 0000000000451f80 CR3: 00000000638f0000 CR4: 00000000000006e0
>>> >> Stack:
>>> >>  0000000000000000 ffff88006bbc0000 ffff88006bbc8000 0000000000000000
>>> >>  0000000000000002 ffff88006b79d800 0000000000000000 ffff88006bbc7f48
>>> >>  ffffffff852adc60 0000000000000000 ffffffff852adc64 1ffffffff0b40135
>>> >> Call Trace:
>>> >>  [<ffffffff8120c5ae>] lock_acquire+0x17e/0x340 kernel/locking/lockdep.c:3746
>>> >>  [<     inline     >] __mutex_lock_common kernel/locking/mutex.c:521
>>> >>  [<ffffffff83fb6fe1>] mutex_lock_nested+0xb1/0x890 kernel/locking/mutex.c:621
>>> >>  [<ffffffff82db6fd0>] netlink_dump+0x50/0xac0 net/netlink/af_netlink.c:2067
>>> >>  [<ffffffff82dba381>] __netlink_dump_start+0x501/0x770
>>> >> net/netlink/af_netlink.c:2200
>>> >>  [<ffffffff82dc35b2>] genl_family_rcv_msg+0xa02/0xc80
>>> >> net/netlink/genetlink.c:595
>>> >>  [<ffffffff82dc39e6>] genl_rcv_msg+0x1b6/0x270 net/netlink/genetlink.c:658
>>> >>  [<ffffffff82dc1a70>] netlink_rcv_skb+0x2c0/0x3b0 net/netlink/af_netlink.c:2281
>>> >>  [<ffffffff82dc2b98>] genl_rcv+0x28/0x40 net/netlink/genetlink.c:669
>>> >>  [<     inline     >] netlink_unicast_kernel net/netlink/af_netlink.c:1214
>>> >>  [<ffffffff82dc0329>] netlink_unicast+0x5a9/0x880 net/netlink/af_netlink.c:1240
>>> >>  [<ffffffff82dc0fb7>] netlink_sendmsg+0x9b7/0xce0 net/netlink/af_netlink.c:1786
>>> >>  [<     inline     >] sock_sendmsg_nosec net/socket.c:606
>>> >>  [<ffffffff82b7075c>] sock_sendmsg+0xcc/0x110 net/socket.c:616
>>> >>  [<ffffffff82b709c1>] sock_write_iter+0x221/0x3b0 net/socket.c:814
>>> >>  [<     inline     >] new_sync_write fs/read_write.c:499
>>> >>  [<ffffffff8151c944>] __vfs_write+0x334/0x570 fs/read_write.c:512
>>> >>  [<ffffffff8152045b>] vfs_write+0x17b/0x500 fs/read_write.c:560
>>> >>  [<     inline     >] SYSC_write fs/read_write.c:607
>>> >>  [<ffffffff81523d84>] SyS_write+0xd4/0x1a0 fs/read_write.c:599
>>> >>  [<ffffffff83fc0141>] entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> >> arch/x86/entry/entry_64.S:209
>>> >> Code: 0f 1f 44 00 00 f6 c4 02 0f 85 24 0a 00 00 44 8b 35 c9 61 8b 03
>>> >> 45 85 f6 74 2c 4c 89 fa 48 bb 00 00 00 00 00 fc ff df 48 c1 ea 03 <80>
>>> >> 3c 1a 00 0f 85 04 2f 00 00 49 81 3f a0 dc 2a 85 41 be 00 00
>>> >> RIP  [<ffffffff8120872d>] __lock_acquire+0x12d/0x3450
>>> >> kernel/locking/lockdep.c:3221
>>> >>  RSP <ffff88006bbc7420>
>>> >> ---[ end trace 685b3c182bf7f25c ]---
>>> >>
>>> >> The reproducer is attached.
>>> >>
>>> >> On commit 1a1891d762d6e64daf07b5be4817e3fbb29e3c59 (Oct 18).
>>> >
>>> > (Adding more maintainers)
>>> >
>>> > Still seeing this on 0c183d92b20b5c84ca655b45ef57b3318b83eb9e (Oct 31).
>>>
>>> Here is another report that might be related:
>>>
>>> =====================================
>>> [ BUG: bad unlock balance detected! ]
>>> 4.9.0-rc3+ #336 Not tainted
>>> -------------------------------------
>>> syz-executor/4018 is trying to release lock ([   36.220068] nl_table_lock
>>> ) at:
>>> [<ffffffff82dc8683>] netlink_diag_dump+0x1a3/0x250 net/netlink/diag.c:182
>>> but there are no more locks to release!
>>>
>>> other info that might help us debug this:
>>> 3 locks held by syz-executor/4018:
>>>  #0: [   36.220068]  (
>>> sock_diag_mutex[   36.220068] ){+.+.+.}
>>> , at: [   36.220068] [<ffffffff82c3873b>] sock_diag_rcv+0x1b/0x40
>>>  #1: [   36.220068]  (
>>> sock_diag_table_mutex[   36.220068] ){+.+.+.}
>>> , at: [   36.220068] [<ffffffff82c38e00>] sock_diag_rcv_msg+0x140/0x3a0
>>>  #2: [   36.220068]  (
>>> nlk->cb_mutex[   36.220068] ){+.+.+.}
>>> , at: [   36.220068] [<ffffffff82db6600>] netlink_dump+0x50/0xac0
>>>
>>> stack backtrace:
>>> CPU: 1 PID: 4018 Comm: syz-executor Not tainted 4.9.0-rc3+ #336
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  ffff8800645df688 ffffffff81b46934 ffffffff84eb3e78 ffff88006ad85800
>>>  ffffffff82dc8683 ffffffff84eb3e78 ffff8800645df6b8 ffffffff812043ca
>>>  dffffc0000000000 ffff88006ad85ff8 ffff88006ad85fd0 00000000ffffffff
>>> Call Trace:
>>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>>  [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
>>>  [<ffffffff812043ca>] print_unlock_imbalance_bug+0x17a/0x1a0
>>> kernel/locking/lockdep.c:3388
>>>  [<     inline     >] __lock_release kernel/locking/lockdep.c:3512
>>>  [<ffffffff8120cfd8>] lock_release+0x8e8/0xc60 kernel/locking/lockdep.c:3765
>>>  [<     inline     >] __raw_read_unlock ./include/linux/rwlock_api_smp.h:225
>>>  [<ffffffff83fc001a>] _raw_read_unlock+0x1a/0x30 kernel/locking/spinlock.c:255
>>>  [<ffffffff82dc8683>] netlink_diag_dump+0x1a3/0x250 net/netlink/diag.c:182
>>>  [<ffffffff82db6947>] netlink_dump+0x397/0xac0 net/netlink/af_netlink.c:2110
>>>  [<ffffffff82db99b1>] __netlink_dump_start+0x501/0x770
>>> net/netlink/af_netlink.c:2200
>>>  [<     inline     >] netlink_dump_start ./include/linux/netlink.h:165
>>>  [<ffffffff82dc75d1>] netlink_diag_handler_dump+0x191/0x220
>>> net/netlink/diag.c:218
>>>  [<     inline     >] __sock_diag_cmd net/core/sock_diag.c:239
>>>  [<ffffffff82c38fd6>] sock_diag_rcv_msg+0x316/0x3a0 net/core/sock_diag.c:270
>>>  [<ffffffff82dc10a0>] netlink_rcv_skb+0x2c0/0x3b0 net/netlink/af_netlink.c:2281
>>>  [<ffffffff82c3874a>] sock_diag_rcv+0x2a/0x40 net/core/sock_diag.c:281
>>>  [<     inline     >] netlink_unicast_kernel net/netlink/af_netlink.c:1214
>>>  [<ffffffff82dbf959>] netlink_unicast+0x5a9/0x880 net/netlink/af_netlink.c:1240
>>>  [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0 net/netlink/af_netlink.c:1786
>>>  [<     inline     >] sock_sendmsg_nosec net/socket.c:606
>>>  [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110 net/socket.c:616
>>>  [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0 net/socket.c:814
>>>  [<     inline     >] new_sync_write fs/read_write.c:499
>>>  [<ffffffff8151bd44>] __vfs_write+0x334/0x570 fs/read_write.c:512
>>>  [<ffffffff8151f85b>] vfs_write+0x17b/0x500 fs/read_write.c:560
>>>  [<     inline     >] SYSC_write fs/read_write.c:607
>>>  [<ffffffff81523184>] SyS_write+0xd4/0x1a0 fs/read_write.c:599
>>>  [<ffffffff83fc0401>] entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> arch/x86/entry/entry_64.S:209
>>> ------------[ cut here ]------------
>>> WARNING: CPU: 1 PID: 4018 at net/core/skbuff.c:654[<        none
>>>  >] skb_release_head_state+0x1ca/0x240 net/core/skbuff.c:654
>>> Modules linked in:
>>> CPU: 1 PID: 4018 Comm: syz-executor Not tainted 4.9.0-rc3+ #336
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  ffff8800645df920 ffffffff81b46934 0000000000000000 0000000000000000
>>>  ffffffff84401fa0 0000000000000000 ffff8800645df968 ffffffff811112f7
>>>  ffffffff83fb92f2 ffff88000000028e ffffffff84401fa0 000000000000028e
>>> Call Trace:
>>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>>  [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
>>>  [<ffffffff811112f7>] __warn+0x1a7/0x1f0 kernel/panic.c:550
>>>  [<ffffffff8111150c>] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>>>  [<ffffffff82b885ea>] skb_release_head_state+0x1ca/0x240 net/core/skbuff.c:654
>>>  [<ffffffff82b91815>] skb_release_all+0x15/0x60 net/core/skbuff.c:668
>>>  [<     inline     >] __kfree_skb net/core/skbuff.c:684
>>>  [<ffffffff82ba0175>] consume_skb+0x115/0x2e0 net/core/skbuff.c:757
>>>  [<     inline     >] netlink_unicast_kernel net/netlink/af_netlink.c:1215
>>>  [<ffffffff82dbf961>] netlink_unicast+0x5b1/0x880 net/netlink/af_netlink.c:1240
>>>  [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0 net/netlink/af_netlink.c:1786
>>>  [<     inline     >] sock_sendmsg_nosec net/socket.c:606
>>>  [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110 net/socket.c:616
>>>  [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0 net/socket.c:814
>>>  [<     inline     >] new_sync_write fs/read_write.c:499
>>>  [<ffffffff8151bd44>] __vfs_write+0x334/0x570 fs/read_write.c:512
>>>  [<ffffffff8151f85b>] vfs_write+0x17b/0x500 fs/read_write.c:560
>>>  [<     inline     >] SYSC_write fs/read_write.c:607
>>>  [<ffffffff81523184>] SyS_write+0xd4/0x1a0 fs/read_write.c:599
>>>  [<ffffffff83fc0401>] entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> arch/x86/entry/entry_64.S:209
>>> ---[ end trace bb9fa7cf182d59a5 ]---
>>> BUG: scheduling while atomic: syz-executor/4018/0x7fffffff
>>> INFO: lockdep is turned off.
>>> Modules linked in:
>>> CPU: 1 PID: 4018 Comm: syz-executor Tainted: G        W       4.9.0-rc3+ #336
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  ffff8800645dfe28 ffffffff81b46934 dffffc0000000000 000000007fffffff
>>>  00000000000214c0 0000000000000001 ffff8800645dfe48 ffffffff8119113a
>>>  ffff88006cd214c0 0000000000000000 ffff8800645dfec8 ffffffff83fb030a
>>> Call Trace:
>>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>>  [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
>>>  [<ffffffff8119113a>] __schedule_bug+0xfa/0x140 kernel/sched/core.c:3230
>>>  [<     inline     >] schedule_debug kernel/sched/core.c:3245
>>>  [<ffffffff83fb030a>] __schedule+0xfda/0x1ab0 kernel/sched/core.c:3345
>>>  [<ffffffff83fb0e70>] schedule+0x90/0x1b0 kernel/sched/core.c:3457
>>>  [<ffffffff810039e9>] exit_to_usermode_loop+0xc9/0x130
>>> arch/x86/entry/common.c:149
>>>  [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:190
>>>  [<ffffffff81006298>] syscall_return_slowpath+0x1a8/0x1e0
>>> arch/x86/entry/common.c:259
>>>  [<ffffffff83fc04a2>] entry_SYSCALL_64_fastpath+0xc0/0xc2
>>> arch/x86/entry/entry_64.S:244
>>> NOHZ: local_softirq_pending 202
>>> ------------[ cut here ]------------
>>> WARNING: CPU: 1 PID: 4018 at net/core/skbuff.c:654[<        none
>>>  >] skb_release_head_state+0x1ca/0x240 net/core/skbuff.c:654
>>> Modules linked in:[   36.328353] CPU: 1 PID: 4018 Comm: syz-executor
>>> Tainted: G        W       4.9.0-rc3+ #336
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  ffff8800645df920 ffffffff81b46934 0000000000000000 0000000000000000
>>>  ffffffff84401fa0 0000000000000000 ffff8800645df968 ffffffff811112f7
>>>  ffffffff83fb92f2 ffff88000000028e ffffffff84401fa0 000000000000028e
>>> Call Trace:
>>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>>  [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
>>>  [<ffffffff811112f7>] __warn+0x1a7/0x1f0 kernel/panic.c:550
>>>  [<ffffffff8111150c>] warn_slowpath_null+0x2c/0x40 kernel/panic.c:585
>>>  [<ffffffff82b885ea>] skb_release_head_state+0x1ca/0x240 net/core/skbuff.c:654
>>>  [<ffffffff82b91815>] skb_release_all+0x15/0x60 net/core/skbuff.c:668
>>>  [<     inline     >] __kfree_skb net/core/skbuff.c:684
>>>  [<ffffffff82ba0175>] consume_skb+0x115/0x2e0 net/core/skbuff.c:757
>>>  [<     inline     >] netlink_unicast_kernel net/netlink/af_netlink.c:1215
>>>  [<ffffffff82dbf961>] netlink_unicast+0x5b1/0x880 net/netlink/af_netlink.c:1240
>>>  [<ffffffff82dc05e7>] netlink_sendmsg+0x9b7/0xce0 net/netlink/af_netlink.c:1786
>>>  [<     inline     >] sock_sendmsg_nosec net/socket.c:606
>>>  [<ffffffff82b6f75c>] sock_sendmsg+0xcc/0x110 net/socket.c:616
>>>  [<ffffffff82b6f9c1>] sock_write_iter+0x221/0x3b0 net/socket.c:814
>>>  [<     inline     >] new_sync_write fs/read_write.c:499
>>>  [<ffffffff8151bd44>] __vfs_write+0x334/0x570 fs/read_write.c:512
>>>  [<ffffffff8151f85b>] vfs_write+0x17b/0x500 fs/read_write.c:560
>>>  [<     inline     >] SYSC_write fs/read_write.c:607
>>>  [<ffffffff81523184>] SyS_write+0xd4/0x1a0 fs/read_write.c:599
>>>  [<ffffffff83fc0401>] entry_SYSCALL_64_fastpath+0x1f/0xc2
>>> arch/x86/entry/entry_64.S:209
>>> ---[ end trace bb9fa7cf182d59a6 ]---
>>> BUG: sleeping function called from invalid context at
>>> ./include/linux/freezer.h:56
>>> in_atomic(): 1, irqs_disabled(): 0, pid: 4018, name: syz-executor
>>> INFO: lockdep is turned off.
>>> CPU: 1 PID: 4018 Comm: syz-executor Tainted: G        W       4.9.0-rc3+ #336
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  ffff8800645dfbb0 ffffffff81b46934 ffff88006ad85800 ffff8800645d8000
>>>  ffff88006ad85800 0000000000000000 ffff8800645dfbd8 ffffffff81192131
>>>  ffff88006ad85800 ffffffff8404c140 0000000000000038 ffff8800645dfc18
>>> Call Trace:
>>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>>  [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
>>>  [<ffffffff81192131>] ___might_sleep+0x281/0x3c0 kernel/sched/core.c:7767
>>>  [<ffffffff81192306>] __might_sleep+0x96/0x1a0 kernel/sched/core.c:7726
>>>  [<     inline     >] try_to_freeze_unsafe ./include/linux/freezer.h:56
>>>  [<     inline     >] try_to_freeze ./include/linux/freezer.h:66
>>>  [<ffffffff81143849>] get_signal+0x129/0x15a0 kernel/signal.c:2147
>>>  [<ffffffff81054aad>] do_signal+0x8d/0x1a30 arch/x86/kernel/signal.c:807
>>>  [<ffffffff81003a05>] exit_to_usermode_loop+0xe5/0x130
>>> arch/x86/entry/common.c:156
>>>  [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:190
>>>  [<ffffffff81006298>] syscall_return_slowpath+0x1a8/0x1e0
>>> arch/x86/entry/common.c:259
>>>  [<ffffffff83fc04a2>] entry_SYSCALL_64_fastpath+0xc0/0xc2
>>> arch/x86/entry/entry_64.S:244
>>> Kernel panic - not syncing: Aiee, killing interrupt handler!
>>> CPU: 1 PID: 4018 Comm: syz-executor Tainted: G        W       4.9.0-rc3+ #336
>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>>  ffff8800645df998 ffffffff81b46934 0000000000000003 dffffc0000000000
>>>  dffffc0000000000 ffff8800645dfa04 ffff8800645dfa60 ffffffff8140bf7a
>>>  0000000041b58ab3 ffffffff84797a7d ffffffff8140bdbe ffffffff00000000
>>> Call Trace:
>>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>>  [<ffffffff81b46934>] dump_stack+0xb3/0x10f lib/dump_stack.c:51
>>>  [<ffffffff8140bf7a>] panic+0x1bc/0x39d kernel/panic.c:179
>>>  [<ffffffff8111cfd8>] do_exit+0x1b48/0x2ac0 kernel/exit.c:740
>>>  [<ffffffff811222be>] do_group_exit+0x10e/0x340 kernel/exit.c:931
>>>  [<ffffffff81143d54>] get_signal+0x634/0x15a0 kernel/signal.c:2307
>>>  [<ffffffff81054aad>] do_signal+0x8d/0x1a30 arch/x86/kernel/signal.c:807
>>>  [<ffffffff81003a05>] exit_to_usermode_loop+0xe5/0x130
>>> arch/x86/entry/common.c:156
>>>  [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:190
>>>  [<ffffffff81006298>] syscall_return_slowpath+0x1a8/0x1e0
>>> arch/x86/entry/common.c:259
>>>  [<ffffffff83fc04a2>] entry_SYSCALL_64_fastpath+0xc0/0xc2
>>> arch/x86/entry/entry_64.S:244
>>> Kernel Offset: disabled
>>> ---[ end Kernel panic - not syncing: Aiee, killing interrupt handler!
>>
>>
>> This is probably a leftover after commit
>> ad202074320cd75b31b8cdb58cca0d4ef6aaea8a
>> ("netlink: Use rhashtable walk interface in diag dump")
>>
>> Please try this trivial fix :
>>
>> diff --git a/net/netlink/diag.c b/net/netlink/diag.c
>> index b2f0e986a6f4..a5546249fb10 100644
>> --- a/net/netlink/diag.c
>> +++ b/net/netlink/diag.c
>> @@ -178,11 +178,8 @@ static int netlink_diag_dump(struct sk_buff *skb, struct netlink_callback *cb)
>>                 }
>>                 cb->args[1] = i;
>>         } else {
>> -               if (req->sdiag_protocol >= MAX_LINKS) {
>> -                       read_unlock(&nl_table_lock);
>> -                       rcu_read_unlock();
>> +               if (req->sdiag_protocol >= MAX_LINKS)
>>                         return -ENOENT;
>> -               }
>>
>>                 err = __netlink_diag_dump(skb, cb, req->sdiag_protocol, s_num);
>>         }
>>
>>

Powered by blists - more mailing lists