lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150604045536.GB30151@wfg-t540p.sh.intel.com>
Date:	Thu, 4 Jun 2015 12:55:36 +0800
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	Frederic Weisbecker <fweisbec@...il.com>
Cc:	fengguang.wu@...el.com, Ingo Molnar <mingo@...nel.org>,
	"Peter Zijlstra (Intel)" <peterz@...radead.org>,
	kpreempt-tech@...ts.sourceforge.net, linux-kernel@...r.kernel.org,
	LKP <lkp@...org>
Subject: [sched/preempt] BUG: kernel boot crashed

Greetings,

0day kernel testing robot got the below dmesg and the first bad commit is

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master

commit b30f0e3ffedfa52b1d67a302ae5860c49998e5e2
Author:     Frederic Weisbecker <fweisbec@...il.com>
AuthorDate: Tue May 12 16:41:49 2015 +0200
Commit:     Ingo Molnar <mingo@...nel.org>
CommitDate: Tue May 19 08:39:12 2015 +0200

    sched/preempt: Optimize preemption operations on __schedule() callers
    
    __schedule() disables preemption and some of its callers
    (the preempt_schedule*() family) also set PREEMPT_ACTIVE.
    
    So we have two preempt_count() modifications that could be performed
    at once.
    
    Lets remove the preemption disablement from __schedule() and pull
    this responsibility to its callers in order to optimize preempt_count()
    operations in a single place.
    
    Suggested-by: Linus Torvalds <torvalds@...ux-foundation.org>
    Signed-off-by: Frederic Weisbecker <fweisbec@...il.com>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
    Cc: Peter Zijlstra <peterz@...radead.org>
    Cc: Thomas Gleixner <tglx@...utronix.de>
    Link: http://lkml.kernel.org/r/1431441711-29753-5-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar <mingo@...nel.org>

+----------------------------------------------------+------------+------------+------------+
|                                                    | 90b62b5129 | b30f0e3ffe | f407a82586 |
+----------------------------------------------------+------------+------------+------------+
| boot_successes                                     | 195        | 0          | 0          |
| boot_failures                                      | 0          | 65         | 100        |
| BUG:kernel_boot_hang                               | 0          | 58         | 76         |
| BUG:kernel_boot_crashed                            | 0          | 4          | 18         |
| BUG:unable_to_handle_kernel                        | 0          | 3          | 6          |
| WARNING:at_lib/kobject.c:#kobject_add_internal()   | 0          | 2          | 6          |
| Oops                                               | 0          | 1          | 6          |
| Oops:#[##]PREEMPT_PREEMPT_x_x                      | 0          | 1          |            |
| EIP_is_at_sock_alloc                               | 0          | 2          |            |
| Kernel_panic-not_syncing:Fatal_exception           | 0          | 2          | 6          |
| backtrace:platform_device_add                      | 0          | 2          | 6          |
| backtrace:regulator_dummy_init                     | 0          | 2          | 6          |
| backtrace:regulator_init                           | 0          | 2          | 6          |
| backtrace:kernel_init_freeable                     | 0          | 2          | 6          |
| backtrace:kset_create_and_add                      | 0          | 2          | 6          |
| backtrace:of_init                                  | 0          | 2          | 6          |
| backtrace:__netlink_kernel_create                  | 0          | 2          |            |
| backtrace:rtnetlink_net_init                       | 0          | 2          |            |
| backtrace:ops_init                                 | 0          | 2          | 6          |
| backtrace:register_pernet_subsys                   | 0          | 2          | 6          |
| backtrace:rtnetlink_init                           | 0          | 2          |            |
| backtrace:netlink_proto_init                       | 0          | 2          | 6          |
| WARNING:at_kernel/trace/trace.c:#register_tracer() | 0          | 0          | 1          |
| WARNING:at_fs/sysfs/dir.c:#sysfs_warn_dup()        | 0          | 0          | 6          |
| WARNING:at_fs/proc/generic.c:#__proc_create()      | 0          | 0          | 6          |
| EIP_is_at_parameq                                  | 0          | 0          | 6          |
| backtrace:__class_register                         | 0          | 0          | 6          |
| backtrace:cpuidle_add_interface                    | 0          | 0          | 6          |
| backtrace:cpuidle_init                             | 0          | 0          | 6          |
| backtrace:proc_mkdir_data                          | 0          | 0          | 6          |
| backtrace:netfilter_net_init                       | 0          | 0          | 6          |
| backtrace:netfilter_init                           | 0          | 0          | 6          |
| backtrace:sock_init                                | 0          | 0          | 6          |
| backtrace:proc_create_data                         | 0          | 0          | 6          |
| backtrace:netlink_net_init                         | 0          | 0          | 6          |
| backtrace:parse_args                               | 0          | 0          | 6          |
+----------------------------------------------------+------------+------------+------------+

[    0.132797] clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 6370867519511994 ns
[    0.133406] Testing tracer function: 
[    0.133406] Testing tracer function: 
BUG: kernel boot crashed
Elapsed time: 10
qemu-system-x86_64 -enable-kvm -cpu kvm64 -kernel /pkg/linux/i386-randconfig-x1-06011148/gcc-4.9/b30f0e3ffedfa52b1d67a302ae5860c49998e5e2/vmlinuz-4.1.0-rc4-00025-gb30f0e3 -append 'hung_task_panic=1 earlyprintk=ttyS0,115200 systemd.log_level=err debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 console=ttyS0,115200 console=tty0 vga=normal  root=/dev/ram0 rw link=/kbuild-tests/run-queue/kvm/i386-randconfig-x1-06011148/tip:sched:core:b30f0e3ffedfa52b1d67a302ae5860c49998e5e2:bisect-linux-6/.vmlinuz-b30f0e3ffedfa52b1d67a302ae5860c49998e5e2-20150603202622-18-ivb41 branch=tip/sched/core BOOT_IMAGE=/pkg/linux/i386-randconfig-x1-06011148/gcc-4.9/b30f0e3ffedfa52b1d67a302ae5860c49998e5e2/vmlinuz-4.1.0-rc4-00025-gb30f0e3 drbd.minor_count=8'  -initrd /osimage/quantal/quantal-core-i386.cgz -m 300 -smp 2 -device e1000,netdev=net0 -netdev user,id=net0 -boot order=nc -no-reboot -watchdog i6300esb -rtc base=localtime -drive file=/fs/vdisk/disk0-quantal-ivb41-47,media=disk,if=virtio -drive file=/fs/vdisk/disk1-quantal-ivb41-47,media=disk,if=virtio -drive file=/fs/vdisk/disk2-quantal-ivb41-47,media=disk,if=virtio -drive file=/fs/vdisk/disk3-quantal-ivb41-47,media=disk,if=virtio -drive file=/fs/vdisk/disk4-quantal-ivb41-47,media=disk,if=virtio -drive file=/fs/vdisk/disk5-quantal-ivb41-47,media=disk,if=virtio -drive file=/fs/vdisk/disk6-quantal-ivb41-47,media=disk,if=virtio -pidfile /dev/shm/kboot/pid-quantal-ivb41-47 -serial file:/dev/shm/kboot/serial-quantal-ivb41-47 -daemonize -display none -monitor null 

git bisect start f407a8258610169cd8e975dba7f0b2824562014c 3e0283a53f7d2f2dae7bc4aa7f3104cb5988018f --
git bisect  bad 70ffdb9393a7264a069265edded729078dcf0425  # 07:24      2-      3  mm/fault, arch: Use pagefault_disable() to check for disabled pagefaults in the handler
git bisect good fa6004ad4528153b699a4d5ce5ea6b33acce74cc  # 20:09     65+      1  ipc/mqueue: Implement lockless pipelined wakeups
git bisect  bad b30f0e3ffedfa52b1d67a302ae5860c49998e5e2  # 20:27      1-      3  sched/preempt: Optimize preemption operations on __schedule() callers
git bisect good a22ae718067c233af790b8690b3d8f6190859ead  # 20:38     65+      0  Merge tag 'v4.1-rc4' into sched/core, before applying new patches
git bisect good 2e10e71ce88e3eaccfd09a045ae6ecebe657ba09  # 21:00     65+      0  sched/preempt: Rearrange a few symbols after headers merge
git bisect good 90b62b5129d5cb50f62f40e684de7a1961e57197  # 21:08     65+      0  sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET
# first bad commit: [b30f0e3ffedfa52b1d67a302ae5860c49998e5e2] sched/preempt: Optimize preemption operations on __schedule() callers
git bisect good 90b62b5129d5cb50f62f40e684de7a1961e57197  # 21:11    195+      0  sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET
# extra tests on HEAD of tip/sched/core
git bisect  bad f407a8258610169cd8e975dba7f0b2824562014c  # 21:11      0-    100  Merge branch 'linus' into sched/core, to resolve conflict
# extra tests on tree/branch next/master
# extra tests with first bad commit reverted
# extra tests on tree/branch linus/master
git bisect good 8cd9234c64c584432f6992fe944ca9e46ca8ea76  # 22:01    195+      0  Merge tag 'please-pull-rusty' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux
# extra tests on tree/branch next/master


This script may reproduce the error.

----------------------------------------------------------------------------
#!/bin/bash

kernel=$1

kvm=(
	qemu-system-x86_64
	-enable-kvm
	-cpu kvm64
	-kernel $kernel
	-m 300
	-smp 2
	-device e1000,netdev=net0
	-netdev user,id=net0
	-boot order=nc
	-no-reboot
	-watchdog i6300esb
	-rtc base=localtime
	-serial stdio
	-display none
	-monitor null 
)

append=(
	hung_task_panic=1
	earlyprintk=ttyS0,115200
	systemd.log_level=err
	debug
	apic=debug
	sysrq_always_enabled
	rcupdate.rcu_cpu_stall_timeout=100
	panic=-1
	softlockup_panic=1
	nmi_watchdog=panic
	oops=panic
	load_ramdisk=2
	prompt_ramdisk=0
	console=ttyS0,115200
	console=tty0
	vga=normal
	root=/dev/ram0
	rw
	drbd.minor_count=8
)

"${kvm[@]}" --append "${append[*]}"
----------------------------------------------------------------------------

Thanks,
Fengguang

View attachment "dmesg-quantal-ivb41-47:20150603202642:i386-randconfig-x1-06011148:4.1.0-rc4-00025-gb30f0e3:1" of type "text/plain" (29585 bytes)

View attachment "config-4.1.0-rc4-00025-gb30f0e3" of type "text/plain" (104427 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ