lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:   Fri, 30 Oct 2020 15:47:40 +0800
From:   kernel test robot <lkp@...el.com>
To:     "Paul E. McKenney" <paulmck@...nel.org>
Cc:     LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        lkp@...el.com
Subject: [rcu] 44bad5b3cc:
 dmesg.INFO:rcu_preempt_detected_stalls_on_CPUs/tasks

Greeting,

FYI, we noticed the following commit (built with gcc-9):

commit: 44bad5b3cca2d452d17ef82841b20b42a2cf11a0 ("rcu: Do full report for .need_qs for strict GPs")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: trinity
version: trinity-i386-4d2343bd-1_20200320
with following parameters:

	runtime: 300s

test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/


on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):


+--------------------------------------------------+------------+------------+
|                                                  | f19920e412 | 44bad5b3cc |
+--------------------------------------------------+------------+------------+
| boot_successes                                   | 23         | 16         |
| boot_failures                                    | 21         | 48         |
| BUG:kernel_hang_in_test_stage                    | 21         | 47         |
| INFO:rcu_preempt_self-detected_stall_on_CPU      | 4          |            |
| EIP:_raw_spin_unlock_irq                         | 5          | 25         |
| EIP:___might_sleep                               | 1          |            |
| EIP:preempt_schedule_common                      | 4          | 2          |
| EIP:preempt_schedule_notrace                     | 4          |            |
| EIP:__put_user_4                                 | 1          |            |
| EIP:schedule                                     | 9          | 40         |
| INFO:rcu_preempt_detected_stalls_on_CPUs/tasks   | 8          | 40         |
| WARNING:at_net/sched/sch_generic.c:#dev_watchdog | 2          | 1          |
| EIP:dev_watchdog                                 | 2          | 1          |
+--------------------------------------------------+------------+------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp@...el.com>


[  311.442723] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  311.445504] 	(detected by 1, t=100002 jiffies, g=83721, q=12)
[  311.446491] rcu: All QSes seen, last rcu_preempt kthread activity 100002 (10023--89979), jiffies_till_next_fqs=3, root ->qsmask 0x0
[  311.448502] rcu: rcu_preempt kthread starved for 100002 jiffies! g83721 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
[  311.450260] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  311.451793] rcu: RCU grace-period kthread stack dump:
[  311.452650] task:rcu_preempt     state:R  running task     stack:    0 pid:   10 ppid:     2 flags:0x00004008
[  311.454190] Call Trace:
[  311.454569]  sched_show_task+0x125/0x137
[  311.455208]  rcu_check_gp_kthread_starvation+0xe4/0xf6
[  311.456058]  rcu_sched_clock_irq+0x7c3/0x89a
[  311.456793]  ? account_system_time+0x68/0x71
[  311.457543]  update_process_times+0x24/0x4f
[  311.458273]  tick_sched_handle+0x55/0x63
[  311.458959]  tick_sched_timer+0x3b/0x79
[  311.459634]  ? tick_sched_do_timer+0x44/0x44
[  311.460353]  __hrtimer_run_queues+0x13a/0x1fe
[  311.461130]  ? tick_sched_do_timer+0x44/0x44
[  311.461861]  hrtimer_interrupt+0xb0/0x1bc
[  311.462575]  ? sysvec_call_function_single+0x2c/0x2c
[  311.463423]  __sysvec_apic_timer_interrupt+0xc3/0x13b
[  311.464306]  sysvec_apic_timer_interrupt+0x1b/0x2e
[  311.465097]  handle_exception+0x143/0x143
[  311.465747] EIP: _raw_spin_unlock_irq+0x25/0x2f
[  311.466519] Code: 9e ff 5b 5d c3 66 66 66 66 90 55 89 e5 53 89 c3 83 c0 10 8b 55 04 e8 e5 80 a2 ff 89 d8 e8 1e b4 a2 ff fb 64 ff 0d f4 06 c8 41 <75> 05 e8 fe 77 98 ff 5b 5d c3 66 66 66 66 90 55 89 e5 56 89 d6 53
[  311.469547] EAX: 00000001 EBX: f57777c0 ECX: 00000000 EDX: f4eab840
[  311.470573] ESI: 00000000 EDI: f4eab840 EBP: f4eb5df8 ESP: f4eb5df4
[  311.471709] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00000202
[  311.472930]  ? sysvec_call_function_single+0x2c/0x2c
[  311.473771]  ? _raw_spin_unlock_irq+0x25/0x2f
[  311.474512]  finish_task_switch+0x151/0x1dd
[  311.475244]  ? finish_task_switch+0x129/0x1dd
[  311.476000]  __schedule+0x4a2/0x4df
[  311.476630]  ? sysvec_call_function_single+0x2c/0x2c
[  311.477479]  preempt_schedule_irq+0x45/0x56
[  311.478204]  irqentry_exit_cond_resched+0x37/0x3a
[  311.479018]  irqentry_exit+0x33/0x42
[  311.479649]  sysvec_apic_timer_interrupt+0x2a/0x2e
[  311.480520]  handle_exception+0x143/0x143
[  311.481187] EIP: schedule+0x8c/0xb2
[  311.481752] Code: 39 d1 74 0a ba 01 00 00 00 e8 e7 de c8 ff 64 8b 35 fc 06 c8 41 64 ff 05 f4 06 c8 41 31 c0 e8 9c fa ff ff 64 ff 0d f4 06 c8 41 <8b> 06 a8 08 75 e5 8b 43 14 a9 20 00 00 20 74 12 a8 20 89 d8 74 07
[  311.484809] EAX: 00000000 EBX: f4eab840 ECX: 00000000 EDX: f4eab840
[  311.485882] ESI: f4eab840 EDI: 00000003 EBP: f4eb5ef8 ESP: f4eb5ef0
[  311.486967] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00000246
[  311.488134]  ? sysvec_call_function_single+0x2c/0x2c
[  311.489016]  ? schedule+0x8c/0xb2
[  311.489606]  schedule_timeout+0x88/0xbf
[  311.490291]  ? __next_timer_interrupt+0xb7/0xb7
[  311.491084]  rcu_gp_kthread+0x545/0x8fa
[  311.491761]  ? call_rcu+0x189/0x189
[  311.492381]  kthread+0x101/0x106
[  311.492923]  ? call_rcu+0x189/0x189
[  311.493522]  ? kthread_queue_delayed_work+0x55/0x55
[  311.494374]  ret_from_fork+0x1c/0x30


To reproduce:

        # build kernel
	cd linux
	cp config-5.9.0-rc1-00006-g44bad5b3cca2d .config
	make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email



Thanks,
lkp


View attachment "config-5.9.0-rc1-00006-g44bad5b3cca2d" of type "text/plain" (105578 bytes)

View attachment "job-script" of type "text/plain" (4258 bytes)

Download attachment "dmesg.xz" of type "application/x-xz" (19016 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ