lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <20150602035012.GA16893@wfg-t540p.sh.intel.com>
Date:	Tue, 2 Jun 2015 11:50:12 +0800
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	Frederic Weisbecker <fweisbec@...il.com>
Cc:	Ingo Molnar <mingo@...nel.org>,
	"Peter Zijlstra (Intel)" <peterz@...radead.org>, LKP <lkp@...org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: [sched/preempt] BUG: scheduling while atomic: swapper/0/0/0x00200002

Hi Frederic,

FYI, here is another bisect result.

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master

commit b30f0e3ffedfa52b1d67a302ae5860c49998e5e2
Author:     Frederic Weisbecker <fweisbec@...il.com>
AuthorDate: Tue May 12 16:41:49 2015 +0200
Commit:     Ingo Molnar <mingo@...nel.org>
CommitDate: Tue May 19 08:39:12 2015 +0200

    sched/preempt: Optimize preemption operations on __schedule() callers
    
    __schedule() disables preemption and some of its callers
    (the preempt_schedule*() family) also set PREEMPT_ACTIVE.
    
    So we have two preempt_count() modifications that could be performed
    at once.
    
    Lets remove the preemption disablement from __schedule() and pull
    this responsibility to its callers in order to optimize preempt_count()
    operations in a single place.
    
    Suggested-by: Linus Torvalds <torvalds@...ux-foundation.org>
    Signed-off-by: Frederic Weisbecker <fweisbec@...il.com>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
    Cc: Peter Zijlstra <peterz@...radead.org>
    Cc: Thomas Gleixner <tglx@...utronix.de>
    Link: http://lkml.kernel.org/r/1431441711-29753-5-git-send-email-fweisbec@gmail.com
    Signed-off-by: Ingo Molnar <mingo@...nel.org>

+-----------------------------------------------------------------------------+------------+------------+---------------+
|                                                                             | 90b62b5129 | b30f0e3ffe | next-20150529 |
+-----------------------------------------------------------------------------+------------+------------+---------------+
| boot_successes                                                              | 123        | 0          | 0             |
| boot_failures                                                               | 0          | 41         | 14            |
| BUG:scheduling_while_atomic                                                 | 0          | 1          | 2             |
| BUG:unable_to_handle_kernel                                                 | 0          | 38         | 9             |
| Oops                                                                        | 0          | 28         | 9             |
| RIP:task_curr                                                               | 0          | 19         |               |
| Kernel_panic-not_syncing:Fatal_exception                                    | 0          | 26         | 10            |
| BUG:kernel_boot_hang                                                        | 0          | 1          | 1             |
| RIP:put_prev_task_fair                                                      | 0          | 1          |               |
| WARNING:at_kernel/sched/core.c:#__might_sleep()                             | 0          | 9          |               |
| RIP:ftrace_push_return_trace                                                | 0          | 3          | 3             |
| WARNING:at_kernel/trace/trace_functions_graph.c:#ftrace_return_to_handler() | 0          | 2          | 2             |
| RIP:print_ftrace_graph_addr                                                 | 0          | 3          | 1             |
| RIP:update_curr                                                             | 0          | 3          | 3             |
| Kernel_panic-not_syncing:Fatal_exception_in_interrupt                       | 0          | 3          |               |
| kernel_BUG_at_arch/x86/kernel/traps.c                                       | 0          | 1          |               |
| invalid_opcode                                                              | 0          | 1          |               |
| RIP:fixup_bad_iret                                                          | 0          | 1          |               |
| general_protection_fault                                                    | 0          | 2          | 1             |
| WARNING:CPU:#PID:#at_kernel/trace/trace_functions_graph.c:##()              | 0          | 1          |               |
| Kernel_panic-not_syncing:stack-protector:Kernel_stack_is_corrupted_in       | 0          | 1          |               |
| WARNING:at_fs/proc/generic.c:#__proc_create()                               | 0          | 1          |               |
| RIP:parameqn                                                                | 0          | 1          |               |
| backtrace:proc_create_data                                                  | 0          | 1          |               |
| backtrace:netlink_net_init                                                  | 0          | 1          |               |
| backtrace:ops_init                                                          | 0          | 1          |               |
| backtrace:register_pernet_subsys                                            | 0          | 1          |               |
| backtrace:netlink_proto_init                                                | 0          | 1          |               |
| backtrace:kernel_init_freeable                                              | 0          | 1          |               |
| backtrace:parse_args                                                        | 0          | 1          |               |
| RIP:vsnprintf                                                               | 0          | 3          |               |
| WARNING:at_include/linux/uaccess.h:#is_prefetch()                           | 0          | 0          | 2             |
| WARNING:at_include/linux/uaccess.h:#return_to_handler()                     | 0          | 0          | 2             |
| WARNING:at_include/linux/uaccess.h:#bad_address()                           | 0          | 0          | 2             |
| RIP:strnlen                                                                 | 0          | 0          | 2             |
| WARNING:at_include/linux/uaccess.h:#show_regs()                             | 0          | 0          | 2             |
| RIP:rb_erase                                                                | 0          | 0          | 1             |
+-----------------------------------------------------------------------------+------------+------------+---------------+

[    0.755065] Testing tracer function_graph: 
[    0.755065] Testing tracer function_graph: 

[    0.801268] BUG: scheduling while atomic: swapper/0/0/0x00200002
[    0.801268] BUG: scheduling while atomic: swapper/0/0/0x00200002
[    0.820350] no locks held by swapper/0/0.
[    0.820350] no locks held by swapper/0/0.
[    0.823335] Preemption disabled at:
[    0.823335] Preemption disabled at:[<ffffffff8300ca98>] return_to_handler+0x0/0x28
[<ffffffff8300ca98>] return_to_handler+0x0/0x28
[    0.828183] 
[    0.828183] 
[    0.874182] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.1.0-rc4-00025-gb30f0e3 #1
[    0.874182] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.1.0-rc4-00025-gb30f0e3 #1
[    0.875191] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[    0.875191] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[    0.876169]  ffffffff841a9e08
[    0.876169]  ffffffff841a9e08 ffffffff841a9e08 ffffffff841a9e08 ffffffff8300ca98 ffffffff8300ca98 ffffffff84221340 ffffffff84221340

[    0.880165]  0000000000000000
[    0.880165]  0000000000000000 ffffffff841a9e38 ffffffff841a9e38 ffffffff81171047 ffffffff81171047 0000000000000000 0000000000000000

[    0.960323]  ffff880010fd5b80
[    0.960323]  ffff880010fd5b80 0000000000000000 0000000000000000 0000000000000000 0000000000000000 ffffffff841a9eb8 ffffffff841a9eb8

[    1.056361] Call Trace:
[    1.056361] Call Trace:
[    1.059166]  <UNK> 
[    1.059166]  <UNK> 

[    1.166614] BUG: unable to handle kernel 
[    1.166614] BUG: unable to handle kernel paging requestpaging request at 00000003a6e9c380
 at 00000003a6e9c380
[    1.167000] IP:
[    1.167000] IP: [<ffffffff8117717b>] task_curr+0x1b/0x40
 [<ffffffff8117717b>] task_curr+0x1b/0x40
[    1.167000] PGD 0 
[    1.167000] PGD 0 

[    1.167000] Thread overran stack, or stack corrupted
[    1.167000] Thread overran stack, or stack corrupted
[    1.167000] Oops: 0000 [#1] 
[    1.167000] Oops: 0000 [#1] PREEMPT PREEMPT SMP SMP DEBUG_PAGEALLOC DEBUG_PAGEALLOC 

[    1.167000] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W       4.1.0-rc4-00025-gb30f0e3 #1
[    1.167000] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W       4.1.0-rc4-00025-gb30f0e3 #1
[    1.167000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[    1.167000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
[    1.167000] task: ffffffff84221340 ti: ffffffff84200000 task.ti: ffffffff84200000
[    1.167000] task: ffffffff84221340 ti: ffffffff84200000 task.ti: ffffffff84200000
[    1.167000] RIP: 0010:[<ffffffff8117717b>] 
[    1.167000] RIP: 0010:[<ffffffff8117717b>]  [<ffffffff8117717b>] task_curr+0x1b/0x40
 [<ffffffff8117717b>] task_curr+0x1b/0x40
[    1.167000] RSP: 0000:ffffffff841a9d50  EFLAGS: 00010002
[    1.167000] RSP: 0000:ffffffff841a9d50  EFLAGS: 00010002
[    1.167000] RAX: 00000000001d5b80 RBX: ffffffff84221340 RCX: 0000000000000002
[    1.167000] RAX: 00000000001d5b80 RBX: ffffffff84221340 RCX: 0000000000000002
[    1.167000] RDX: 00000000842000d8 RSI: ffffffff84221340 RDI: ffffffff84221340
[    1.167000] RDX: 00000000842000d8 RSI: ffffffff84221340 RDI: ffffffff84221340
[    1.167000] RBP: ffffffff841a9da8 R08: ffffffff84221db0 R09: 0000000000000001
[    1.167000] RBP: ffffffff841a9da8 R08: ffffffff84221db0 R09: 0000000000000001
[    1.167000] R10: 0000000000000050 R11: 0000000000000003 R12: 0000000000000004
[    1.167000] R10: 0000000000000050 R11: 0000000000000003 R12: 0000000000000004
[    1.167000] R13: ffffffff84223440 R14: 0000000000000005 R15: 0000000000000000
[    1.167000] R13: ffffffff84223440 R14: 0000000000000005 R15: 0000000000000000
[    1.167000] FS:  0000000000000000(0000) GS:ffff880010e00000(0000) knlGS:0000000000000000
[    1.167000] FS:  0000000000000000(0000) GS:ffff880010e00000(0000) knlGS:0000000000000000
[    1.167000] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    1.167000] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    1.167000] CR2: 00000003a6e9c380 CR3: 000000000421a000 CR4: 00000000000006b4
[    1.167000] CR2: 00000003a6e9c380 CR3: 000000000421a000 CR4: 00000000000006b4
[    1.167000] Stack:
[    1.167000] Stack:
[    1.167000]  ffffffff8112f01b
[    1.167000]  ffffffff8112f01b ffffffff8112ed05 ffffffff8112ed05 0000000000000000 0000000000000000 ffffffff841a9da8 ffffffff841a9da8

[    1.167000]  ffffffff816070b3
[    1.167000]  ffffffff816070b3 ffffffff86454b20 ffffffff86454b20 0000000000000000 0000000000000000 0000000000000005 0000000000000005

[    1.167000]  ffffffff841a9e58
[    1.167000]  ffffffff841a9e58 ffffffff84221340 ffffffff84221340 0000000000000002 0000000000000002 ffffffff841a9e08 ffffffff841a9e08

[    1.167000] Call Trace:
[    1.167000] Call Trace:
[    1.167000]  <UNK> 
[    1.167000]  <UNK> 

[    1.167000] Code: 
[    1.167000] Code: e8 e8 eb eb fd fd ff ff ff ff 48 48 83 83 05 05 03 03 28 28 2b 2b 05 05 01 01 5d 5d c3 c3 90 90 48 48 8b 8b 57 57 08 08 55 55 48 48 c7 c7 c0 c0 80 80 5b 5b 1d 1d 00 00 48 48 83 83 05 05 24 24 29 29 2b 2b 05 05 01 01 8b 8b 52 52 10 10 48 48 89 89 e5 e5 5d 5d <48> <48> 8b 8b 14 14 d5 d5 c0 c0 bc bc e9 e9 85 85 48 48 39 39 bc bc 10 10 e0 e0 08 08 00 00 00 00 0f 0f 94 94 c0 c0 0f 0f b6 b6 

[    1.167000] RIP 
[    1.167000] RIP  [<ffffffff8117717b>] task_curr+0x1b/0x40
 [<ffffffff8117717b>] task_curr+0x1b/0x40
[    1.167000]  RSP <ffffffff841a9d50>
[    1.167000]  RSP <ffffffff841a9d50>
[    1.167000] CR2: 00000003a6e9c380
[    1.167000] CR2: 00000003a6e9c380
[    1.167000] ---[ end trace 6e322970baec994b ]---
[    1.167000] ---[ end trace 6e322970baec994b ]---

git bisect start 7732a9817fb01002bde7615066e86c156fb5a31b ba155e2d21f6bf05de86a78dbe5bfd8757604a65 --
git bisect good e7900c6f171ee82868cf0dcf398cf3b8f44114f1  # 12:47     25+      2  Merge remote-tracking branch 'thermal-soc/next'
git bisect  bad c566f03d886fe9b6979b0a4e9a49aa61bfa3dd0c  # 12:53      4-     22  Merge remote-tracking branch 'clockevents/clockevents/next'
git bisect good 02fe2b99dfd80f254c8c491a9c00b5fc42a2eae7  # 13:02     25+      1  Merge remote-tracking branch 'device-mapper/for-next'
git bisect good b89ac4e6de3d2234180cccbf25e23979d1673c9f  # 13:12     25+      2  Merge remote-tracking branch 'security/next'
git bisect good cd8133cd99ac0863dbbd2920ceba899218844fbd  # 13:21     25+      4  Merge remote-tracking branch 'trivial/for-next'
git bisect good e0aca8d773e99485aefe3115396f92fd4fbd238b  # 13:29     25+      0  Merge remote-tracking branch 'mailbox/mailbox-for-next'
git bisect good 342b07b3e19807e536755c7394bc9eb0ade5341c  # 13:38     25+      5  Merge remote-tracking branch 'spi/for-next'
git bisect  bad b114f50dde045c71fa1833ddabe2b70c52576f90  # 13:44      5-     22  Merge remote-tracking branch 'tip/auto-latest'
git bisect  bad c41f3eb84e58355b9209c961328067e2c23a1aeb  # 13:50     11-     25  Merge branch 'timers/nohz'
git bisect good 17186ccda374ae02ef231cbbc8f1825e7c19ddbd  # 13:58     27+      0  perf/x86/intel: Make WARN()ings consistent
git bisect  bad b7794610cdd2533e303de1027c1a4d9576875e51  # 14:12     17-     25  manual merge of sched/core
git bisect  bad 06931e62246844c73fba24d7aeb4a5dc897a2739  # 14:19      3-     40  sched/topology: Rename topology_thread_cpumask() to topology_sibling_cpumask()
git bisect  bad 8bcbde5480f9777f8b74d71493722c663e22c21b  # 14:26     23-     41  sched/preempt, mm/fault: Count pagefault_disable() levels in pagefault_disabled
git bisect good 7110744516276e906f9197e2857d026eb2343393  # 02:10     41+      4  sched, timer: Use the atomic task_cputime in thread_group_cputimer
git bisect good a22ae718067c233af790b8690b3d8f6190859ead  # 02:28     41+     10  Merge tag 'v4.1-rc4' into sched/core, before applying new patches
git bisect good 90b62b5129d5cb50f62f40e684de7a1961e57197  # 02:36     41+      0  sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET
git bisect  bad e017cf21ae82e0b36f026b22083a8ae67926f465  # 02:43      0-     34  sched/preempt: Fix out of date comment
git bisect  bad b30f0e3ffedfa52b1d67a302ae5860c49998e5e2  # 02:53      2-     40  sched/preempt: Optimize preemption operations on __schedule() callers
# first bad commit: [b30f0e3ffedfa52b1d67a302ae5860c49998e5e2] sched/preempt: Optimize preemption operations on __schedule() callers
git bisect good 90b62b5129d5cb50f62f40e684de7a1961e57197  # 02:57    123+      0  sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET
# extra tests with DEBUG_INFO
git bisect  bad b30f0e3ffedfa52b1d67a302ae5860c49998e5e2  # 03:05      7-    109  sched/preempt: Optimize preemption operations on __schedule() callers
# extra tests on HEAD of next/master
git bisect  bad 7732a9817fb01002bde7615066e86c156fb5a31b  # 03:15      0-     14  Add linux-next specific files for 20150529
# extra tests on tree/branch next/master
# extra tests with first bad commit reverted
# extra tests on tree/branch linus/master
git bisect good c65b99f046843d2455aa231747b5a07a999a9f3d  # 04:38    123+     21  Linux 4.1-rc6
# extra tests on tree/branch next/master


This script may reproduce the error.

----------------------------------------------------------------------------
#!/bin/bash

kernel=$1

kvm=(
	qemu-system-x86_64
	-enable-kvm
	-cpu kvm64
	-kernel $kernel
	-m 300
	-smp 2
	-device e1000,netdev=net0
	-netdev user,id=net0
	-boot order=nc
	-no-reboot
	-watchdog i6300esb
	-rtc base=localtime
	-serial stdio
	-display none
	-monitor null 
)

append=(
	hung_task_panic=1
	earlyprintk=ttyS0,115200
	systemd.log_level=err
	debug
	apic=debug
	sysrq_always_enabled
	rcupdate.rcu_cpu_stall_timeout=100
	panic=-1
	softlockup_panic=1
	nmi_watchdog=panic
	oops=panic
	load_ramdisk=2
	prompt_ramdisk=0
	console=ttyS0,115200
	console=tty0
	vga=normal
	root=/dev/ram0
	rw
	drbd.minor_count=8
)

"${kvm[@]}" --append "${append[*]}"
----------------------------------------------------------------------------

Thanks,
Fengguang

View attachment "dmesg-quantal-vp-10:20150602025329:x86_64-randconfig-h1-06011129:4.1.0-rc4-00025-gb30f0e3:1" of type "text/plain" (51728 bytes)

View attachment "config-4.1.0-rc4-00025-gb30f0e3" of type "text/plain" (97974 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ