| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 22 Apr 2019 08:17:38 +0800
From: kernel test robot <rong.a.chen@...el.com>
To: Jonathan Lemon <jonathan.lemon@...il.com>
Cc: netdev@...r.kernel.org, dsahern@...il.com, kernel-team@...com,
lkp@...org
Subject: [route] 3be5fb4873: INFO:rcu_sched_self-detected_stall_on_CPU
FYI, we noticed the following commit (built with gcc-7):
commit: 3be5fb4873d0c2b007a33813aa3be487aa478199 ("[PATCH net] route: Avoid crash from dereferencing NULL rt->from")
url: https://github.com/0day-ci/linux/commits/Jonathan-Lemon/route-Avoid-crash-from-dereferencing-NULL-rt-from/20190415-143536
in testcase: rcutorture
with following parameters:
runtime: 300s
test: default
torture_type: tasks
test-description: rcutorture is rcutorture kernel module load/unload test.
test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------+------------+------------+
| | 7324880182 | 3be5fb4873 |
+-------------------------------------------------+------------+------------+
| boot_successes | 6 | 6 |
| boot_failures | 0 | 10 |
| BUG:kernel_reboot-without-warning_in_test_stage | 0 | 4 |
| INFO:rcu_sched_self-detected_stall_on_CPU | 0 | 3 |
| RIP:e1000_xmit_frame | 0 | 3 |
| INFO:rcu_tasks_detected_stalls_on_tasks | 0 | 1 |
| RIP:rcu_gp_kthread | 0 | 1 |
| INFO:rcu_sched_detected_stalls_on_CPUs/tasks | 0 | 3 |
| RIP:default_idle | 0 | 1 |
| RIP:_raw_spin_unlock_irqrestore | 0 | 1 |
+-------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp@...el.com>
[ 433.373241] rcu: INFO: rcu_sched self-detected stall on CPU
[ 433.373241] rcu: 1-...!: (1 ticks this GP) idle=396/1/0x4000000000000002 softirq=24930/24930 fqs=0
[ 1303.421117] rcu: (t=217466 jiffies g=59689 q=10)
[ 1303.452767] rcu: rcu_sched kthread starved for 217466 jiffies! g59689 f0x0 RCU_GP_INIT(4) ->state=0x0 ->cpu=1
[ 1303.510249] rcu: RCU grace-period kthread stack dump:
[ 1303.510249] rcu_sched R running task 0 10 2 0x80000008
[ 1303.510249] Call Trace:
[ 1303.510249] <IRQ>
[ 1303.510249] sched_show_task+0x110/0x142
[ 1303.510249] rcu_check_gp_kthread_starvation+0x9f/0xab
[ 1303.801293] rcu_sched_clock_irq+0x268/0x5ab
[ 1303.801293] ? __accumulate_pelt_segments+0x29/0x3a
[ 1303.801293] ? tick_sched_handle+0x3e/0x3e
[ 1303.801293] update_process_times+0x28/0x4a
[ 1303.801293] tick_sched_handle+0x34/0x3e
[ 1303.801293] tick_sched_timer+0x3b/0x69
[ 1303.801293] __hrtimer_run_queues+0x16e/0x22c
[ 1303.801293] ? kvm_clock_read+0x14/0x1c
[ 1303.801293] hrtimer_interrupt+0xf9/0x212
[ 1303.801293] smp_apic_timer_interrupt+0xd0/0x13a
[ 1303.801293] apic_timer_interrupt+0xf/0x20
[ 1303.801293] </IRQ>
[ 1303.801293] RIP: 0010:rcu_gp_kthread+0x343/0x875
[ 1304.241071] Code: b0 43 01 ff c8 39 c2 75 11 48 8b 53 08 4c 89 e1 48 89 de e8 8b ed ff ff eb 10 c6 03 00 e8 4f 33 05 00 fb 65 ff 0d 50 0d e7 7e <65> 48 8b 04 25 80 4d 01 00 8a 90 f0 02 00 00 84 d2 74 07 c6 80 f0
[ 1304.241071] RSP: 0000:ffffc90000ca3ed0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
[ 1304.241071] RAX: 0000000080000002 RBX: ffffffff8246ae00 RCX: 3fffffffffff122d
[ 1304.241071] RDX: 0000000080000002 RSI: 0000000000000000 RDI: ffffffff811a3fb8
[ 1304.241071] RBP: 0000000000022200 R08: 0000000080000001 R09: 0000000000000000
[ 1304.241071] R10: ffffc90001113dd0 R11: ffffc90001113b20 R12: 0000000000000246
[ 1304.241071] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff811a3c7d
[ 1304.241071] ? rcu_fwd_progress_check+0x115/0x115
[ 1304.241071] ? rcu_gp_kthread+0x33b/0x875
[ 1304.241071] kthread+0x121/0x129
[ 1304.241071] ? kthread_park+0x76/0x76
[ 1304.241071] ret_from_fork+0x3a/0x50
[ 1304.241071] NMI backtrace for cpu 1
[ 1304.241071] CPU: 1 PID: 10 Comm: rcu_sched Not tainted 5.1.0-rc4-00123-g3be5fb4 #2
[ 1304.241071] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 1304.241071] Call Trace:
[ 1304.241071] <IRQ>
[ 1304.241071] dump_stack+0x67/0x8e
[ 1304.241071] nmi_cpu_backtrace+0x71/0x83
[ 1304.241071] ? lapic_can_unplug_cpu+0x93/0x93
[ 1306.330098] nmi_trigger_cpumask_backtrace+0x5e/0xe3
[ 1306.345384] rcu_dump_cpu_stacks+0x91/0xb9
[ 1306.345384] rcu_sched_clock_irq+0x26d/0x5ab
[ 1306.345384] ? __accumulate_pelt_segments+0x29/0x3a
[ 1306.345384] ? tick_sched_handle+0x3e/0x3e
[ 1306.345384] update_process_times+0x28/0x4a
[ 1306.345384] tick_sched_handle+0x34/0x3e
[ 1306.345384] tick_sched_timer+0x3b/0x69
[ 1306.345384] __hrtimer_run_queues+0x16e/0x22c
[ 1306.345384] ? kvm_clock_read+0x14/0x1c
[ 1306.345384] hrtimer_interrupt+0xf9/0x212
[ 1306.345384] smp_apic_timer_interrupt+0xd0/0x13a
[ 1306.345384] apic_timer_interrupt+0xf/0x20
[ 1306.345384] </IRQ>
[ 1306.345384] RIP: 0010:rcu_gp_kthread+0x343/0x875
[ 1306.345384] Code: b0 43 01 ff c8 39 c2 75 11 48 8b 53 08 4c 89 e1 48 89 de e8 8b ed ff ff eb 10 c6 03 00 e8 4f 33 05 00 fb 65 ff 0d 50 0d e7 7e <65> 48 8b 04 25 80 4d 01 00 8a 90 f0 02 00 00 84 d2 74 07 c6 80 f0
[ 1306.345384] RSP: 0000:ffffc90000ca3ed0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
[ 1306.345384] RAX: 0000000080000002 RBX: ffffffff8246ae00 RCX: 3fffffffffff122d
[ 1306.345384] RDX: 0000000080000002 RSI: 0000000000000000 RDI: ffffffff811a3fb8
[ 1306.345384] RBP: 0000000000022200 R08: 0000000080000001 R09: 0000000000000000
[ 1306.345384] R10: ffffc90001113dd0 R11: ffffc90001113b20 R12: 0000000000000246
[ 1306.345384] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff811a3c7d
[ 1306.345384] ? rcu_fwd_progress_check+0x115/0x115
[ 1306.345384] ? rcu_gp_kthread+0x33b/0x875
[ 1306.345384] kthread+0x121/0x129
[ 1306.345384] ? kthread_park+0x76/0x76
[ 1306.345384] ret_from_fork+0x3a/0x50
Starting Cleanup of Temporary Directories...
[ 1310.495777] tasks-torture:--- Start of test: nreaders=1 nfakewriters=4 stat_interval=60 verbose=1 test_no_idle_hz=1 shuffle_interval=3 stutter=5 irqreader=1 fqs_duration=0 fqs_holdoff=0 fqs_stutter=3 test_boost=1/0 test_boost_interval=7 test_boost_duration=4 shutdown_secs=0 stall_cpu=0 stall_cpu_holdoff=10 stall_cpu_irqsoff=0 n_barrier_cbs=0 onoff_interval=0 onoff_holdoff=0
[ 1310.835122] tasks-torture: Creating rcu_torture_writer task
[ 1312.087846] tasks-torture: Creating rcu_torture_fakewriter task
[ 1312.232201] tasks-torture: rcu_torture_writer task started
[ 1312.271956] rcu_torture_writer: Testing expedited GPs.
[ 1312.391586] rcu_torture_writer: Testing asynchronous GPs.
[
[ 1312.501542] rcu_torture_writer: Testing normal GPs.
[ 1313.218748] tasks-torture: rcu_torture_fakewriter task started
[ 1313.474662] tasks-torture: Creating rcu_torture_fakewriter task
[ 1313.597156] tasks-torture: rcu_torture_fakewriter task started
[ 1314.092357] tasks-torture: Creating rcu_torture_fakewriter task
[ 1314.331123] tasks-torture: rcu_torture_fakewriter task started
Starting watchdog daemon...
[ 1314.815156] tasks-torture: Creating rcu_torture_reader task
[ 1314.817143] tasks-torture: rcu_torture_fakewriter task started
[ 1315.117809] tasks-torture: Creating rcu_torture_stats task
[ 1315.290265] tasks-torture: rcu_torture_reader task started
[ 1315.472111] tasks-torture: Creating torture_shuffle task
[ 1315.607589] tasks-torture: rcu_torture_stats task started
[ 1316.004729] tasks-torture: Creating torture_stutter task
[ 1316.109365] tasks-torture: torture_shuffle task started
[ 1316.349951] tasks-torture: torture_stutter task started
Starting Update UTMP about System Runlevel Changes...
[ 1338.941389] random: crng init done
[ 1338.941389] random: 2 urandom warning(s) missed due to ratelimiting
[ 1377.751185] tasks-torture: rtc: 000000003d6fec49 ver: 1 tfle: 0 rta: 2 rtaf: 0 rtf: 0 rtmbe: 0 rtbe: 0 rtbke: 0 rtbre: 0 rtbf: 0 rtb: 0 nt: 156 onoff: 0/0:0/0 -1,0:-1,0 0:0 (HZ=250) barrier: 0/0:0
[ 1377.812382] tasks-torture: Reader Pipe: 299081 0 0 0 0 0 0 0 0 0 0
[ 1377.836340] tasks-torture: Reader Batch: 299081 0 0 0 0 0 0 0 0 0 0
To reproduce:
# build kernel
cd linux
cp config-5.1.0-rc4-00123-g3be5fb4 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
View attachment "config-5.1.0-rc4-00123-g3be5fb4" of type "text/plain" (109610 bytes)
View attachment "job-script" of type "text/plain" (4808 bytes)
Download attachment "dmesg.xz" of type "application/x-xz" (16724 bytes)
View attachment "rcutorture" of type "text/plain" (5672 bytes)
Powered by blists - more mailing lists