lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Thu, 14 Jan 2021 15:45:11 +0800
From:   kernel test robot <oliver.sang@...el.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     "Paul E. McKenney" <paulmck@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>, lkp@...ts.01.org,
        lkp@...el.com
Subject: [workqueue]  d5bff968ea:
 WARNING:at_kernel/workqueue.c:#process_one_work


Greeting,

FYI, we noticed the following commit (built with gcc-9):

commit: d5bff968ea9cc005e632d9369c26cbd8148c93d5 ("workqueue: break affinity initiatively")
https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git dev.2021.01.11b


in testcase: rcutorture
version: 
with following parameters:

	runtime: 300s
	test: cpuhotplug
	torture_type: srcud

test-description: rcutorture is rcutorture kernel module load/unload test.
test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt


on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):


+--------------------------------------------------+------------+------------+
|                                                  | 6211b34f6e | d5bff968ea |
+--------------------------------------------------+------------+------------+
| boot_successes                                   | 4          | 0          |
| boot_failures                                    | 0          | 12         |
| WARNING:at_kernel/workqueue.c:#process_one_work  | 0          | 12         |
| EIP:process_one_work                             | 0          | 12         |
| WARNING:at_kernel/kthread.c:#kthread_set_per_cpu | 0          | 4          |
| EIP:kthread_set_per_cpu                          | 0          | 4          |
+--------------------------------------------------+------------+------------+


If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang@...el.com>


[   73.794288] WARNING: CPU: 0 PID: 22 at kernel/workqueue.c:2192 process_one_work (kbuild/src/consumer/kernel/workqueue.c:2192) 
[   73.795012] Modules linked in: rcutorture torture mousedev evbug input_leds led_class psmouse pcspkr tiny_power_button button
[   73.795949] CPU: 0 PID: 22 Comm: kworker/1:0 Not tainted 5.11.0-rc3-gd5bff968ea9c #2
[   73.796592] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[   73.797280] Workqueue:  0x0 (rcu_gp)
[   73.797592] EIP: process_one_work (kbuild/src/consumer/kernel/workqueue.c:2192) 
[ 73.797980] Code: 37 64 a1 58 54 4c 43 39 45 24 74 2c 31 c9 ba 01 00 00 00 c7 04 24 01 00 00 00 b8 08 1d f5 42 e8 94 85 13 00 ff 05 b8 30 04 43 <0f> 0b ba 01 00 00 00 eb 22 8d 74 26 00 90 c7 04 24 01 00 00 00 31
All code
========
   0:	37                   	(bad)  
   1:	64 a1 58 54 4c 43 39 	movabs %fs:0x74244539434c5458,%eax
   8:	45 24 74 
   b:	2c 31                	sub    $0x31,%al
   d:	c9                   	leaveq 
   e:	ba 01 00 00 00       	mov    $0x1,%edx
  13:	c7 04 24 01 00 00 00 	movl   $0x1,(%rsp)
  1a:	b8 08 1d f5 42       	mov    $0x42f51d08,%eax
  1f:	e8 94 85 13 00       	callq  0x1385b8
  24:	ff 05 b8 30 04 43    	incl   0x430430b8(%rip)        # 0x430430e2
  2a:*	0f 0b                	ud2    		<-- trapping instruction
  2c:	ba 01 00 00 00       	mov    $0x1,%edx
  31:	eb 22                	jmp    0x55
  33:	8d 74 26 00          	lea    0x0(%rsi,%riz,1),%esi
  37:	90                   	nop
  38:	c7 04 24 01 00 00 00 	movl   $0x1,(%rsp)
  3f:	31                   	.byte 0x31

Code starting with the faulting instruction
===========================================
   0:	0f 0b                	ud2    
   2:	ba 01 00 00 00       	mov    $0x1,%edx
   7:	eb 22                	jmp    0x2b
   9:	8d 74 26 00          	lea    0x0(%rsi,%riz,1),%esi
   d:	90                   	nop
   e:	c7 04 24 01 00 00 00 	movl   $0x1,(%rsp)
  15:	31                   	.byte 0x31
[   73.799597] EAX: 42f51d08 EBX: 00000000 ECX: 00000000 EDX: 00000001
[   73.800143] ESI: 43c04720 EDI: df4140f4 EBP: de7f23c0 ESP: 43d7bf08
[   73.800688] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010002
[   73.801260] CR0: 80050033 CR2: 00aabcbc CR3: 7f9dc000 CR4: 000406d0
[   73.801784] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[   73.802305] DR6: fffe0ff0 DR7: 00000400
[   73.802651] Call Trace:
[   73.802871] ? process_one_work (kbuild/src/consumer/kernel/workqueue.c:615 kbuild/src/consumer/kernel/workqueue.c:643 kbuild/src/consumer/kernel/workqueue.c:2248) 
[   73.803232] ? worker_thread (kbuild/src/consumer/include/linux/list.h:282 (discriminator 1) kbuild/src/consumer/kernel/workqueue.c:2424 (discriminator 1)) 
[   73.803567] ? kthread (kbuild/src/consumer/kernel/kthread.c:292 (discriminator 1)) 
[   73.803859] ? create_worker (kbuild/src/consumer/kernel/workqueue.c:2366) 
[   73.804192] ? kzalloc (kbuild/src/consumer/kernel/resource.c:166) 
[   73.804554] ? ret_from_fork (kbuild/src/consumer/arch/x86/entry/entry_32.S:856) 
[   73.804875] ---[ end trace 122a61cfdd814ce9 ]---
[   73.805580] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   73.883292] smpboot: CPU 1 is now offline
[   73.951310] x86: Booting SMP configuration:
[   73.951830] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   73.952546] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   73.952580] masked ExtINT on CPU#1
[   73.989426] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   74.164842] smpboot: CPU 1 is now offline
[   74.231330] x86: Booting SMP configuration:
[   74.231866] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   74.232581] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   74.232617] masked ExtINT on CPU#1
[   74.262076] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   74.433467] smpboot: CPU 1 is now offline
[   74.591312] x86: Booting SMP configuration:
[   74.591816] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   74.592510] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   74.592546] masked ExtINT on CPU#1
[   74.641675] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   74.803293] smpboot: CPU 1 is now offline
[   74.841947] x86: Booting SMP configuration:
[   74.842481] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   74.843206] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   74.843243] masked ExtINT on CPU#1
[   74.891673] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   74.973277] smpboot: CPU 1 is now offline
[   75.121309] x86: Booting SMP configuration:
[   75.121836] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   75.122539] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   75.122573] masked ExtINT on CPU#1
[   75.171673] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   75.303292] smpboot: CPU 1 is now offline
[   75.341537] x86: Booting SMP configuration:
[   75.342087] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   75.342822] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   75.342859] masked ExtINT on CPU#1
[   75.401637] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   75.523225] smpboot: CPU 1 is now offline
[   75.631293] x86: Booting SMP configuration:
[   75.631790] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   75.632512] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   75.632544] masked ExtINT on CPU#1
[   75.681631] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   75.813300] smpboot: CPU 1 is now offline
[   75.861498] x86: Booting SMP configuration:
[   75.862040] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   75.862763] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   75.862797] masked ExtINT on CPU#1
[   75.911656] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   76.043254] smpboot: CPU 1 is now offline
[   76.091495] x86: Booting SMP configuration:
[   76.092038] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   76.092726] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   76.092758] masked ExtINT on CPU#1
[   76.141672] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   76.273386] smpboot: CPU 1 is now offline
[   76.321583] x86: Booting SMP configuration:
[   76.322137] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   76.323186] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   76.323221] masked ExtINT on CPU#1
[   76.361613] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   76.453350] smpboot: CPU 1 is now offline
[   76.681296] x86: Booting SMP configuration:
[   76.681792] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   76.682448] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   76.682479] masked ExtINT on CPU#1
[   76.731666] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   76.883233] smpboot: CPU 1 is now offline
[   77.191301] x86: Booting SMP configuration:
[   77.191829] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   77.192554] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   77.192589] masked ExtINT on CPU#1
[   77.241673] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   77.323278] smpboot: CPU 1 is now offline
[   77.361452] x86: Booting SMP configuration:
[   77.361957] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   77.362632] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   77.362668] masked ExtINT on CPU#1
[   77.411641] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   77.583568] smpboot: CPU 1 is now offline
[   77.691303] x86: Booting SMP configuration:
[   77.691851] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   77.692549] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   77.692581] masked ExtINT on CPU#1
[   77.751809] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   78.032676] srcud-torture: rtc: (ptrval) ver: 922 tfle: 0 rta: 922 rtaf: 0 rtf: 913 rtmbe: 0 rtmbkf: 0/0 rtbe: 0 rtbke: 0 rtbre: 0 rtbf: 0 rtb: 0 nt: 552 onoff: 93/93:93/94 2,21:0,21 667:787 (HZ=100) barrier: 0/0:0 read-exits: 67 nocb-toggles: 0:0
[   78.035048] srcud-torture: Reader Pipe:  12631032 44 0 0 0 0 0 0 0 0 0
[   78.035759] srcud-torture: Reader Batch:  12630957 119 0 0 0 0 0 0 0 0 0
[   78.036507] srcud-torture: Free-Block Circulation:  921 921 920 919 918 917 916 915 914 913 0
[   78.037407] rcu: srcud-torture: Tree SRCU g6668 per-CPU(idx=1): 0(17,12 .) 1(-17,-12 .) T(0,0)
[   78.055649] smpboot: CPU 1 is now offline
[   78.121332] x86: Booting SMP configuration:
[   78.121837] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   78.150644] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   78.150702] masked ExtINT on CPU#1
[   78.172354] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0
[   78.343316] smpboot: CPU 1 is now offline
[   78.411398] x86: Booting SMP configuration:
[   78.411921] smpboot: Booting Node 0 Processor 1 APIC 0x1
[   78.412640] kvm-clock: cpu 1, msr 34f6021, secondary cpu clock
[   78.412675] masked ExtINT on CPU#1
[   78.437957] kvm-guest: stealtime: cpu 1, msr 9e7e6ec0


To reproduce:

        # build kernel
	cd linux
	cp config-5.11.0-rc3-gd5bff968ea9c .config
	make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email



Thanks,
Oliver Sang


View attachment "config-5.11.0-rc3-gd5bff968ea9c" of type "text/plain" (139930 bytes)

View attachment "job-script" of type "text/plain" (4433 bytes)

Download attachment "dmesg.xz" of type "application/x-xz" (38128 bytes)

View attachment "rcutorture" of type "text/plain" (7385 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ