lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:	Mon, 4 Feb 2008 07:45:54 -0800 (PST)
From:	Doug Kehn <rdkehn@...oo.com>
To:	linux-kernel@...r.kernel.org, uClinux <uclinux-dev@...inux.org>
Subject: Soft lockup 2.6.23.14-uc0

Hi All,

I am observing kernel soft lockups when running
network throughput tests with NUTTCP.  The kernel is a
stock 2.6.23 kernel with patches from uClinux.org.  I
have applied the incremental 2.6.23 patches to produce
the resulting 2.6.23.14-uc0 kernel.  This kernel is
executing on a 266MHz Intel XScale IXP420 processor
with 16MB flash (JFFS2) and 64MB RAM.  I am also using
the Intel Access Library v2.4 with patches from
snapgear.org.  (The Intel Access Library is the reason
for the tainted kernel.)  The toolchain to build the
kernel and all applications is comprised of:

binutils-2.16.tar.gz
gcc-3.4.4.tar.gz
glibc-2.3.3.tar.gz
glibc-linuxthreads-2.3.3.tar.gz

All applications are compiled against uClibc-0.9.27.

A soft lockup dump is provided below.  Any help in
determining the cause of the soft lock will be
appreciated.

Regards,
...doug


# BUG: soft lockup - CPU#0 stuck for 11s! [awk:2960]

Pid: 2960, comm:                  awk
CPU: 0    Tainted: P         (2.6.23.14-uc0 #1)
PC is at handle_IRQ_event+0x34/0x80
LR is at handle_level_irq+0x98/0xec
pc : [<c0058b30>]    lr : [<c0059eb0>]    psr:
40000013
sp : c353deb0  ip : c353ded0  fp : c353decc
r10: 4000d090  r9 : c353c000  r8 : 4000515c
r7 : 00000012  r6 : 00000000  r5 : 00000000  r4 :
c3f68a60
r3 : 40000013  r2 : c025151c  r1 : c3f68a60  r0 :
00000012
Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM 
Segment user
Control: 000039ff  Table: 03500000  DAC: 00000015
[<c0020b60>] (show_regs+0x0/0x4c) from [<c005865c>]
(softlockup_tick+0xe8/0x114)
 r4:00001e13
[<c0058574>] (softlockup_tick+0x0/0x114) from
[<c00401b0>] (run_local_timers+0x1
8/0x1c)
[<c0040198>] (run_local_timers+0x0/0x1c) from
[<c00401f8>] (update_process_times
+0x44/0x6c)
[<c00401b4>] (update_process_times+0x0/0x6c) from
[<c00512c0>] (tick_periodic+0x
94/0xc0)
 r5:c0257900 r4:00989680
[<c005122c>] (tick_periodic+0x0/0xc0) from
[<c0051308>] (tick_handle_periodic+0x
1c/0xc4)
 r5:00000000 r4:00000001
[<c00512ec>] (tick_handle_periodic+0x0/0xc4) from
[<c0029c7c>] (ixp4xx_timer_int
errupt+0x28/0x38)
[<c0029c54>] (ixp4xx_timer_interrupt+0x0/0x38) from
[<c0058b40>] (handle_IRQ_eve
nt+0x44/0x80)
 r4:c0242734
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353df08 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000005 r4:c0244bd8
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353de68 to 0xc353deb0)
de60:                   00000012 c3f68a60 c025151c
40000013 c3f68a60 00000000
de80: 00000000 00000012 4000515c c353c000 4000d090
c353decc c353ded0 c353deb0
dea0: c0059eb0 c0058b30 40000013 ffffffff
 r7:00000012 r6:00000028 r5:0000001f r4:ffffffff
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353dfb0 r6:c025a5c0 r5:00000012 r4:c0244eb0
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000012 r4:c0244eb0
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353df08 to 0xc353df50)
df00:                   ffbee004 c353c000 00000100
20000013 00000022 00000005
df20: 0000000a c0256300 4000515c c353c000 4000d090
c353df6c c353df70 c353df50
df40: c003c34c c003c29c 20000013 ffffffff
 r7:c0256300 r6:00040008 r5:0000001f r4:ffffffff
[<c003c258>] (__do_softirq2+0x0/0xc8) from
[<c003c34c>] (__do_softirq+0x2c/0x34)
 r7:00000000 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c003c320>] (__do_softirq+0x0/0x34) from [<c003c518>]
(irq_exit+0x40/0x48)
[<c003c4d8>] (irq_exit+0x0/0x48) from [<c001f048>]
(asm_do_IRQ+0x48/0x5c)
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fbc0>]
(__irq_usr+0x40/0x80)
Exception stack(0xc353dfb0 to 0xc353dff8)
dfa0:                                     000000f0
0000007a 00000000 00008138
dfc0: 40005060 00008aa4 4000513c 00009fe4 4000515c
40016ee4 4000d090 00000000
dfe0: 00000002 bed765b4 40000ccc 40000dd8 80000010
ffffffff
 r7:00009fe4 r6:00000020 r5:0000001f r4:ffffffff
BUG: soft lockup - CPU#0 stuck for 11s! [awk:2960]

Pid: 2960, comm:                  awk
CPU: 0    Tainted: P         (2.6.23.14-uc0 #1)
PC is at handle_IRQ_event+0x34/0x80
LR is at handle_level_irq+0x98/0xec
pc : [<c0058b30>]    lr : [<c0059eb0>]    psr:
40000013
sp : c353deb0  ip : c353ded0  fp : c353decc
r10: 4000d090  r9 : c353c000  r8 : 4000515c
r7 : 00000012  r6 : 00000000  r5 : 00000000  r4 :
c3f68a60
r3 : 40000013  r2 : c025151c  r1 : c3f68a60  r0 :
00000012
Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM 
Segment user
Control: 000039ff  Table: 03500000  DAC: 00000015
[<c0020b60>] (show_regs+0x0/0x4c) from [<c005865c>]
(softlockup_tick+0xe8/0x114)
 r4:00001e1e
[<c0058574>] (softlockup_tick+0x0/0x114) from
[<c00401b0>] (run_local_timers+0x1
8/0x1c)
[<c0040198>] (run_local_timers+0x0/0x1c) from
[<c00401f8>] (update_process_times
+0x44/0x6c)
[<c00401b4>] (update_process_times+0x0/0x6c) from
[<c00512c0>] (tick_periodic+0x
94/0xc0)
 r5:c0257900 r4:00989680
[<c005122c>] (tick_periodic+0x0/0xc0) from
[<c0051308>] (tick_handle_periodic+0x
1c/0xc4)
 r5:00000000 r4:00000001
[<c00512ec>] (tick_handle_periodic+0x0/0xc4) from
[<c0029c7c>] (ixp4xx_timer_int
errupt+0x28/0x38)
[<c0029c54>] (ixp4xx_timer_interrupt+0x0/0x38) from
[<c0058b40>] (handle_IRQ_eve
nt+0x44/0x80)
 r4:c0242734
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353df08 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000005 r4:c0244bd8
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353de68 to 0xc353deb0)
de60:                   00000012 c3f68a60 c025151c
40000013 c3f68a60 00000000
de80: 00000000 00000012 4000515c c353c000 4000d090
c353decc c353ded0 c353deb0
dea0: c0059eb0 c0058b30 40000013 ffffffff
 r7:00000012 r6:00000028 r5:0000001f r4:ffffffff
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353dfb0 r6:c025a5c0 r5:00000012 r4:c0244eb0
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000012 r4:c0244eb0
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353df08 to 0xc353df50)
df00:                   ffbee004 c353c000 00000100
20000013 00000022 00000005
df20: 0000000a c0256300 4000515c c353c000 4000d090
c353df6c c353df70 c353df50
df40: c003c34c c003c29c 20000013 ffffffff
 r7:c0256300 r6:00040008 r5:0000001f r4:ffffffff
[<c003c258>] (__do_softirq2+0x0/0xc8) from
[<c003c34c>] (__do_softirq+0x2c/0x34)
 r7:00000000 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c003c320>] (__do_softirq+0x0/0x34) from [<c003c518>]
(irq_exit+0x40/0x48)
[<c003c4d8>] (irq_exit+0x0/0x48) from [<c001f048>]
(asm_do_IRQ+0x48/0x5c)
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fbc0>]
(__irq_usr+0x40/0x80)
Exception stack(0xc353dfb0 to 0xc353dff8)
dfa0:                                     000000f0
0000007a 00000000 00008138
dfc0: 40005060 00008aa4 4000513c 00009fe4 4000515c
40016ee4 4000d090 00000000
dfe0: 00000002 bed765b4 40000ccc 40000dd8 80000010
ffffffff
 r7:00009fe4 r6:00000020 r5:0000001f r4:ffffffff
BUG: soft lockup - CPU#0 stuck for 11s! [awk:2960]

Pid: 2960, comm:                  awk
CPU: 0    Tainted: P         (2.6.23.14-uc0 #1)
PC is at __aeabi_idiv+0x3c/0xcc
LR is at do_gettimeofday+0x108/0x130
pc : [<c00fa418>]    lr : [<c004eae0>]    psr:
20000013
sp : c353ddc4  ip : 34bd5d15  fp : c353ddfc
r10: a6e5d944  r9 : 0000007f  r8 : ffffffff
r7 : 00000000  r6 : 00f24ddf  r5 : 00000000  r4 :
000ef02f
r3 : 34bd5efd  r2 : 00000016  r1 : 000003e8  r0 :
00000014
Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM 
Segment user
Control: 000039ff  Table: 03500000  DAC: 00000015
[<c0020b60>] (show_regs+0x0/0x4c) from [<c005865c>]
(softlockup_tick+0xe8/0x114)
 r4:00001e29
[<c0058574>] (softlockup_tick+0x0/0x114) from
[<c00401b0>] (run_local_timers+0x1
8/0x1c)
[<c0040198>] (run_local_timers+0x0/0x1c) from
[<c00401f8>] (update_process_times
+0x44/0x6c)
[<c00401b4>] (update_process_times+0x0/0x6c) from
[<c00512c0>] (tick_periodic+0x
94/0xc0)
 r5:c0257900 r4:00989680
[<c005122c>] (tick_periodic+0x0/0xc0) from
[<c0051308>] (tick_handle_periodic+0x
1c/0xc4)
 r5:00000000 r4:00000001
[<c00512ec>] (tick_handle_periodic+0x0/0xc4) from
[<c0029c7c>] (ixp4xx_timer_int
errupt+0x28/0x38)
[<c0029c54>] (ixp4xx_timer_interrupt+0x0/0x38) from
[<c0058b40>] (handle_IRQ_eve
nt+0x44/0x80)
 r4:c0242734
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353de68 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000005 r4:c0244bd8
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353dd7c to 0xc353ddc4)
dd60:                                                 
              00000014
dd80: 000003e8 00000016 34bd5efd 000ef02f 00000000
00f24ddf 00000000 ffffffff
dda0: 0000007f a6e5d944 c353ddfc 34bd5d15 c353ddc4
c004eae0 c00fa418 20000013
ddc0: ffffffff
 r7:00000000 r6:00000020 r5:0000001f r4:ffffffff
[<c004e9d8>] (do_gettimeofday+0x0/0x130) from
[<bf0ad708>] (0xbf0ad708)
[<bf0ad6f4>] (0xbf0ad6f4) from [<c0058b40>]
(handle_IRQ_event+0x44/0x80)
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353df08 r6:c025a5c0 r5:00000003 r4:c0244b68
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000003 r4:c0244b68
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353de68 to 0xc353deb0)
de60:                   00000012 c3f68a60 c025151c
40000013 c3f68a60 00000000
de80: 00000000 00000012 4000515c c353c000 4000d090
c353decc c353ded0 c353deb0
dea0: c0059eb0 c0058b30 40000013 ffffffff
 r7:00000012 r6:00000008 r5:0000001f r4:ffffffff
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353dfb0 r6:c025a5c0 r5:00000012 r4:c0244eb0
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000012 r4:c0244eb0
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353df08 to 0xc353df50)
df00:                   ffbee004 c353c000 00000100
20000013 00000022 00000005
df20: 0000000a c0256300 4000515c c353c000 4000d090
c353df6c c353df70 c353df50
df40: c003c34c c003c29c 20000013 ffffffff
 r7:c0256300 r6:00040008 r5:0000001f r4:ffffffff
[<c003c258>] (__do_softirq2+0x0/0xc8) from
[<c003c34c>] (__do_softirq+0x2c/0x34)
 r7:00000000 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c003c320>] (__do_softirq+0x0/0x34) from [<c003c518>]
(irq_exit+0x40/0x48)
[<c003c4d8>] (irq_exit+0x0/0x48) from [<c001f048>]
(asm_do_IRQ+0x48/0x5c)
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fbc0>]
(__irq_usr+0x40/0x80)
Exception stack(0xc353dfb0 to 0xc353dff8)
dfa0:                                     000000f0
0000007a 00000000 00008138
dfc0: 40005060 00008aa4 4000513c 00009fe4 4000515c
40016ee4 4000d090 00000000
dfe0: 00000002 bed765b4 40000ccc 40000dd8 80000010
ffffffff
 r7:00009fe4 r6:00000020 r5:0000001f r4:ffffffff
BUG: soft lockup - CPU#0 stuck for 11s! [awk:2960]

Pid: 2960, comm:                  awk
CPU: 0    Tainted: P         (2.6.23.14-uc0 #1)
PC is at __aeabi_idiv+0x54/0xcc
LR is at do_gettimeofday+0x108/0x130
pc : [<c00fa430>]    lr : [<c004eae0>]    psr:
80000013
sp : c353ddc4  ip : 03440146  fp : c353ddfc
r10: d6cc15eb  r9 : 0000007f  r8 : ffffffff
r7 : 00000000  r6 : 00f24de0  r5 : 00000000  r4 :
000ee69f
r3 : 000012ae  r2 : 00000010  r1 : 00003e80  r0 :
0000d600
Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM 
Segment user
Control: 000039ff  Table: 03500000  DAC: 00000015
[<c0020b60>] (show_regs+0x0/0x4c) from [<c005865c>]
(softlockup_tick+0xe8/0x114)
 r4:00001e34
[<c0058574>] (softlockup_tick+0x0/0x114) from
[<c00401b0>] (run_local_timers+0x1
8/0x1c)
[<c0040198>] (run_local_timers+0x0/0x1c) from
[<c00401f8>] (update_process_times
+0x44/0x6c)
[<c00401b4>] (update_process_times+0x0/0x6c) from
[<c00512c0>] (tick_periodic+0x
94/0xc0)
 r5:c0257900 r4:00989680
[<c005122c>] (tick_periodic+0x0/0xc0) from
[<c0051308>] (tick_handle_periodic+0x
1c/0xc4)
 r5:00000000 r4:00000001
[<c00512ec>] (tick_handle_periodic+0x0/0xc4) from
[<c0029c7c>] (ixp4xx_timer_int
errupt+0x28/0x38)
[<c0029c54>] (ixp4xx_timer_interrupt+0x0/0x38) from
[<c0058b40>] (handle_IRQ_eve
nt+0x44/0x80)
 r4:c0242734
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353de68 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000005 r4:c0244bd8
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353dd7c to 0xc353ddc4)
dd60:                                                 
              0000d600
dd80: 00003e80 00000010 000012ae 000ee69f 00000000
00f24de0 00000000 ffffffff
dda0: 0000007f d6cc15eb c353ddfc 03440146 c353ddc4
c004eae0 c00fa430 80000013
ddc0: ffffffff
 r7:00000000 r6:00000020 r5:0000001f r4:ffffffff
[<c004e9d8>] (do_gettimeofday+0x0/0x130) from
[<bf0ad708>] (0xbf0ad708)
[<bf0ad6f4>] (0xbf0ad6f4) from [<c0058b40>]
(handle_IRQ_event+0x44/0x80)
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353df08 r6:c025a5c0 r5:00000003 r4:c0244b68
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000003 r4:c0244b68
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353de68 to 0xc353deb0)
de60:                   00000012 c3f68a60 c025151c
40000013 c3f68a60 00000000
de80: 00000000 00000012 4000515c c353c000 4000d090
c353decc c353ded0 c353deb0
dea0: c0059eb0 c0058b30 40000013 ffffffff
 r7:00000012 r6:00000008 r5:0000001f r4:ffffffff
[<c0058afc>] (handle_IRQ_event+0x0/0x80) from
[<c0059eb0>] (handle_level_irq+0x9
8/0xec)
 r7:c353dfb0 r6:c025a5c0 r5:00000012 r4:c0244eb0
[<c0059e18>] (handle_level_irq+0x0/0xec) from
[<c001f044>] (asm_do_IRQ+0x44/0x5c
)
 r5:00000012 r4:c0244eb0
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fa04>]
(__irq_svc+0x24/0x60)
Exception stack(0xc353df08 to 0xc353df50)
df00:                   ffbee004 c353c000 00000100
20000013 00000022 00000005
df20: 0000000a c0256300 4000515c c353c000 4000d090
c353df6c c353df70 c353df50
df40: c003c34c c003c29c 20000013 ffffffff
 r7:c0256300 r6:00040008 r5:0000001f r4:ffffffff
[<c003c258>] (__do_softirq2+0x0/0xc8) from
[<c003c34c>] (__do_softirq+0x2c/0x34)
 r7:00000000 r6:c025a5c0 r5:00000005 r4:c0244bd8
[<c003c320>] (__do_softirq+0x0/0x34) from [<c003c518>]
(irq_exit+0x40/0x48)
[<c003c4d8>] (irq_exit+0x0/0x48) from [<c001f048>]
(asm_do_IRQ+0x48/0x5c)
[<c001f000>] (asm_do_IRQ+0x0/0x5c) from [<c001fbc0>]
(__irq_usr+0x40/0x80)
Exception stack(0xc353dfb0 to 0xc353dff8)
dfa0:                                     000000f0
0000007a 00000000 00008138
dfc0: 40005060 00008aa4 4000513c 00009fe4 4000515c
40016ee4 4000d090 00000000
dfe0: 00000002 bed765b4 40000ccc 40000dd8 80000010
ffffffff
 r7:00009fe4 r6:00000020 r5:0000001f r4:ffffffff




      ____________________________________________________________________________________
Looking for last minute shopping deals?  
Find them fast with Yahoo! Search.  http://tools.search.yahoo.com/newsearch/category.php?category=shopping
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ