| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 6 Mar 2015 14:07:26 +0800
From: Fengguang Wu <fengguang.wu@...el.com>
To: Dave Chinner <david@...morbit.com>
Cc: fengguang.wu@...el.com, Josef Bacik <jbacik@...com>,
LKP <lkp@...org>, linux-fsdevel@...r.kernel.org,
linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: [bdi] BUG: unable to handle kernel NULL pointer dereference at
0000000000000550
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git superblock-scaling
commit 40ceea09e84d1b9319236b27ad3162422310e5d0
Author: Dave Chinner <dchinner@...hat.com>
AuthorDate: Wed Mar 4 14:36:27 2015 -0500
Commit: Josef Bacik <jbacik@...com>
CommitDate: Wed Mar 4 14:39:28 2015 -0500
bdi: add a new writeback list for sync
wait_sb_inodes() current does a walk of all inodes in the filesystem
to find dirty one to wait on during sync. This is highly
inefficient and wastes a lot of CPU when there are lots of clean
cached inodes that we don't need to wait on.
To avoid this "all inode" walk, we need to track inodes that are
currently under writeback that we need to wait for. We do this by
adding inodes to a writeback list on the bdi when the mapping is
first tagged as having pages under writeback. wait_sb_inodes() can
then walk this list of "inodes under IO" and wait specifically just
for the inodes that the current sync(2) needs to wait for.
To avoid needing all the realted locking to be safe against
interrupts, Jan Kara suggested that we be lazy about removal from
the writeback list. That is, we don't remove inodes from the
writeback list on IO completion, but do it directly during a
wait_sb_inodes() walk.
This means that the a rare sync(2) call will have some work to do
skipping clean inodes However, in the current problem case of
concurrent sync workloads, concurrent wait_sb_inodes() calls only
walk the very recently dispatched inodes and hence should have very
little work to do.
This also means that we have to remove the inodes from the writeback
list during eviction. Do this in inode_wait_for_writeback() once
all writeback on the inode is complete.
Signed-off-by: Dave Chinner <dchinner@...hat.com>
+------------------------------------------+------------+------------+------------+
| | d2ee191143 | 40ceea09e8 | 45b8e7be56 |
+------------------------------------------+------------+------------+------------+
| boot_successes | 72 | 0 | 0 |
| boot_failures | 8 | 20 | 12 |
| BUG:kernel_boot_hang | 8 | | |
| BUG:unable_to_handle_kernel | 0 | 20 | 12 |
| Oops | 0 | 20 | 12 |
| RIP:blk_get_backing_dev_info | 0 | 20 | 12 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 20 | 12 |
| backtrace:add_disk | 0 | 20 | 12 |
| backtrace:brd_init | 0 | 20 | 12 |
| backtrace:kernel_init_freeable | 0 | 20 | 12 |
+------------------------------------------+------------+------------+------------+
[ 0.699779] Linux agpgart interface v0.103
[ 0.700296] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
[ 0.701079] [drm] Initialized drm 1.1.0 20060810
[ 0.702193] BUG: unable to handle kernel NULL pointer dereference at 0000000000000550
[ 0.702883] IP: [<ffffffff8121a84b>] blk_get_backing_dev_info+0xb/0x1a
[ 0.703443] PGD 0
[ 0.703632] Oops: 0000 [#1] PREEMPT SMP
[ 0.704009] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.0.0-rc2-00135-g40ceea0 #1
[ 0.704142] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[ 0.704142] task: ffff880012860000 ti: ffff880012868000 task.ti: ffff880012868000
[ 0.704142] RIP: 0010:[<ffffffff8121a84b>] [<ffffffff8121a84b>] blk_get_backing_dev_info+0xb/0x1a
[ 0.704142] RSP: 0000:ffff88001286bcd8 EFLAGS: 00010202
[ 0.704142] RAX: 0000000000000000 RBX: ffff8800124147a0 RCX: 0000000000000000
[ 0.704142] RDX: ffff880012860720 RSI: 0000000000000000 RDI: ffff8800124145c0
[ 0.704142] RBP: ffff88001286bcd8 R08: 0000000000000001 R09: 0000000000000000
[ 0.704142] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88001282a000
[ 0.704142] R13: ffffffff81c1c600 R14: ffffffff81c1c600 R15: ffff8800124145d8
[ 0.704142] FS: 0000000000000000(0000) GS:ffff880013a00000(0000) knlGS:0000000000000000
[ 0.704142] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 0.704142] CR2: 0000000000000550 CR3: 0000000002211000 CR4: 00000000000006f0
[ 0.704142] Stack:
[ 0.704142] ffff88001286bcf8 ffffffff81155424 ffff8800124147a0 ffff880012414820
[ 0.704142] ffff88001286bd28 ffffffff8115558d ffff880012860720 ffff8800124147a0
[ 0.704142] ffff880012414958 ffffffff81c1c600 ffff88001286bd58 ffffffff8114a2e6
[ 0.704142] Call Trace:
[ 0.704142] [<ffffffff81155424>] inode_to_bdi+0x36/0x45
[ 0.704142] [<ffffffff8115558d>] inode_wait_for_writeback+0x3f/0xc2
[ 0.704142] [<ffffffff8114a2e6>] evict+0xa2/0x15e
[ 0.704142] [<ffffffff8114b084>] iput+0x160/0x16d
[ 0.704142] [<ffffffff8115f8c0>] bdput+0xd/0xf
[ 0.704142] [<ffffffff8115fa28>] __blkdev_put+0x166/0x18a
[ 0.704142] [<ffffffff8116060c>] blkdev_put+0x114/0x11d
[ 0.704142] [<ffffffff81229627>] add_disk+0x44d/0x461
[ 0.704142] [<ffffffff82583a11>] brd_init+0x95/0x160
[ 0.704142] [<ffffffff8258397c>] ? ramdisk_size+0x1a/0x1a
[ 0.704142] [<ffffffff8255205b>] do_one_initcall+0xe8/0x175
[ 0.704142] [<ffffffff825522b8>] kernel_init_freeable+0x1d0/0x258
[ 0.704142] [<ffffffff81b46d9c>] ? rest_init+0xbc/0xbc
[ 0.704142] [<ffffffff81b46da5>] kernel_init+0x9/0xd5
[ 0.704142] [<ffffffff81b5d2fc>] ret_from_fork+0x7c/0xb0
[ 0.704142] [<ffffffff81b46d9c>] ? rest_init+0xbc/0xbc
[ 0.704142] Code: ca 48 c1 ea 04 29 d0 ba 01 00 00 00 89 8f 80 08 00 00 ff c8 85 c0 0f 4e c2 89 87 84 08 00 00 c3 48 8b 87 10 01 00 00 55 48 89 e5 <48> 8b 80 50 05 00 00 5d 48 05 58 02 00 00 c3 48 89 fa 31 c0 b9
[ 0.704142] RIP [<ffffffff8121a84b>] blk_get_backing_dev_info+0xb/0x1a
[ 0.704142] RSP <ffff88001286bcd8>
[ 0.704142] CR2: 0000000000000550
[ 0.704142] ---[ end trace 5c64cf25111d3d67 ]---
[ 0.704142] Kernel panic - not syncing: Fatal exception
git bisect start 45b8e7be563c57fc42d69d5239b4829b5586620d 13a7a6ac0a11197edcd0f756a035f472b42cdf8b --
git bisect bad 980171ac3db20fc792b9b1298067344725a5a285 # 19:07 0- 20 Merge 'luto/x86/entry' into devel-xian-x86_64-201503051818
git bisect bad 7a2a5fad21b95990713cbdfaccc9eeba4e98f9b8 # 19:13 0- 20 Merge 'kees/format-security' into devel-xian-x86_64-201503051818
git bisect good cadb5884edc7353ecb245cf0874ead1f9565f2a7 # 19:29 20+ 0 Merge 'trace/ftrace/urgent' into devel-xian-x86_64-201503051818
git bisect good 30abe812fb9b18b25ebb9d2d214a70013a191ccb # 19:34 20+ 0 Merge 'paulburton/wip-ci20-v4.0' into devel-xian-x86_64-201503051818
git bisect good 0d0fc17147f433ffe27f8d2fcd3b29e109694fe3 # 19:40 20+ 0 Merge 'arm-soc/next/drivers' into devel-xian-x86_64-201503051818
git bisect bad caca114c0271d4df06e2ff1acee68dd62be43d66 # 20:03 0- 20 Merge 'josef-btrfs/superblock-scaling' into devel-xian-x86_64-201503051818
git bisect good d2ee19114357bdf21c59a3ac61eb053ef1c0dc4e # 20:15 20+ 8 inode: rename i_wb_list to i_io_list
git bisect bad 63738525a6ebdf74bb3eb1c3dba16c0bb6895d97 # 20:28 0- 20 inode: convert per-sb inode list to a list_lru
git bisect bad a05899067cddc24276e43e0d440da791738cf967 # 20:42 0- 20 writeback: periodically trim the writeback list
git bisect bad 40ceea09e84d1b9319236b27ad3162422310e5d0 # 21:12 0- 20 bdi: add a new writeback list for sync
# first bad commit: [40ceea09e84d1b9319236b27ad3162422310e5d0] bdi: add a new writeback list for sync
git bisect good d2ee19114357bdf21c59a3ac61eb053ef1c0dc4e # 21:14 60+ 8 inode: rename i_wb_list to i_io_list
# extra tests with DEBUG_INFO
git bisect bad 40ceea09e84d1b9319236b27ad3162422310e5d0 # 22:55 0- 22 bdi: add a new writeback list for sync
# extra tests on HEAD of linux-devel/devel-xian-x86_64-201503051818
git bisect bad 45b8e7be563c57fc42d69d5239b4829b5586620d # 22:55 0- 12 0day head guard for 'devel-xian-x86_64-201503051818'
# extra tests on tree/branch josef-btrfs/superblock-scaling
git bisect bad d119f33d7f868e92c2d7fd21da1aade94584994d # 23:13 0- 60 inode: don't softlockup when evicting inodes
# extra tests on tree/branch linus/master
git bisect good 6587457b4b3d663b237a0f95ddf6e67d1828c8ea # 23:41 60+ 2 Merge tag 'dma-buf-for-4.0-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/sumits/dma-buf
# extra tests on tree/branch next/master
git bisect good cbbf783608bd1f177fd8b1f6498bb2481116beed # 23:53 60+ 0 Add linux-next specific files for 20150305
This script may reproduce the error.
----------------------------------------------------------------------------
#!/bin/bash
kernel=$1
initrd=yocto-minimal-x86_64.cgz
wget --no-clobber https://github.com/fengguang/reproduce-kernel-bug/raw/master/initrd/$initrd
kvm=(
qemu-system-x86_64
-cpu kvm64
-enable-kvm
-kernel $kernel
-initrd $initrd
-m 320
-smp 1
-net nic,vlan=1,model=e1000
-net user,vlan=1
-boot order=nc
-no-reboot
-watchdog i6300esb
-rtc base=localtime
-serial stdio
-display none
-monitor null
)
append=(
hung_task_panic=1
earlyprintk=ttyS0,115200
rd.udev.log-priority=err
systemd.log_target=journal
systemd.log_level=warning
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
console=ttyS0,115200
console=tty0
vga=normal
root=/dev/ram0
rw
drbd.minor_count=8
)
"${kvm[@]}" --append "${append[*]}"
----------------------------------------------------------------------------
Thanks,
Fengguang
View attachment "dmesg-yocto-client7-18:20150305211151:x86_64-acpi-redef:4.0.0-rc2-00135-g40ceea0:1" of type "text/plain" (29149 bytes)
Powered by blists - more mailing lists