| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 11 Dec 2009 14:16:45 -0800
From: Kevin Constantine <kevin.constantine@...il.com>
To: Eric Dumazet <eric.dumazet@...il.com>
CC: netdev@...r.kernel.org
Subject: Re: Kernel Panics in the network stack
On 12/11/2009 01:58 PM, Eric Dumazet wrote:
> Le 11/12/2009 22:50, Kevin Constantine a écrit :
>> On 12/11/2009 01:39 PM, Eric Dumazet wrote:
>>> Le 11/12/2009 22:09, Kevin Constantine a écrit :
>>>> Hey Everyone-
>>>>
>>>> I've been playing with an ARM based linuxstamp
>>>> http://opencircuits.com/Linuxstamp, and I've been seeing kernel panics
>>>> with both 2.6.28.3, and 2.6.30 within an hour or so of turning the
>>>> linuxstamp on. The stack traces always seem to point at functions
>>>> related to networking. I've pasted a couple of the crash outputs below.
>>>> The linuxstamp isn't typically doing anything when the crashes occur,
>>>> in fact it'll crash even if I haven't logged in.
>>>>
>>>> If I ifconfig the interface down, the linuxstamp stays up indefinitely.
>>>> Any pointers in one direction or another would be much appreciated.
>>>>
>>>> I'm not sure if this is the right audience to help out or if the arm
>>>> lists might be better. But in any event, any help would be really
>>>> appreciated.
>>>>
>>>>
>>>> linuxstamp login: Unable to handle kernel paging request at virtual
>>>> address 183cb7b0
>>>> pgd = c0004000
>>>> [183cb7b0] *pgd=00000000
>>>> Internal error: Oops: 0 [#1] PREEMPT
>>>> Modules linked in:
>>>> CPU: 0 Not tainted (2.6.30-00002-g0148992 #13)
>>>> PC is at 0x183cb7b0
>>>> LR is at __udp4_lib_rcv+0x43c/0x72c
>>>
>>> Could you disassemble your vmlinux file, __udp4_lib_rcv function
>>> around LR
>>> <c024ff4c>, to see which function was called ? This function then called
>>> a wrong pointer (0x183cb7b0 not a kernel pointer)
>>>
>>> Maybe a kernel stack corruption, or bad ram, ...
>>
>> The vmlinux file I'm using has probably changed a number of times since
>> then. I'll get a fresh stack trace and disassemble that one.
>>
>> I've has this type of problem with several linuxstamps. I'm only able
>> to trigger this panic when the linuxstamp is plugged into a cisco
>> catalyst gigabit switch. Plugging it in at home into a consumer grade
>> 10/100 switch, the linuxstamp stays up indefinitely.
>>
>> Also worth noting, I'm seeing the error counts in ifconfig increase
>> steadily.
>
> Could be an error in NIC driver error path, this is a good point.
>
>>
>> eth0 Link encap:Ethernet HWaddr ac:de:48:00:00:01
>> inet addr:172.30.194.255 Bcast:172.30.207.255 Mask:255.255.240.0
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:42492 errors:1442 dropped:0 overruns:6 frame:784
>> TX packets:1169 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:3804651 (3.6 MiB) TX bytes:106728 (104.2 KiB)
>> Interrupt:24 Base address:0xc000
>>
>>
>
> Please give us more information, since this platform is not well known :)
>
> lsmod
> cat /proc/meminfo
debian:~# cat /proc/meminfo
MemTotal: 28732 kB
MemFree: 10260 kB
Buffers: 2304 kB
Cached: 9220 kB
SwapCached: 0 kB
Active: 8964 kB
Inactive: 4796 kB
Active(anon): 2292 kB
Inactive(anon): 0 kB
Active(file): 6672 kB
Inactive(file): 4796 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 16 kB
Writeback: 0 kB
AnonPages: 2256 kB
Mapped: 3308 kB
Slab: 3876 kB
SReclaimable: 1508 kB
SUnreclaim: 2368 kB
PageTables: 156 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 14364 kB
Committed_AS: 32372 kB
VmallocTotal: 989184 kB
VmallocUsed: 1136 kB
VmallocChunk: 986108 kB
> cat /proc/cpuinfo
debian:~# cat /proc/cpuinfo
Processor : ARM920T rev 0 (v4l)
BogoMIPS : 89.53
Features : swp half thumb
CPU implementer : 0x41
CPU architecture: 4T
CPU variant : 0x1
CPU part : 0x920
CPU revision : 0
Hardware : emQbit's ECB_AT91
Revision : 0000
Serial : 0000000000000000
> cat /proc/slabinfo (after more than 2000 error count in ifconfig eth0)
debian:~# cat /proc/slabinfo
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab>
<pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata
<active_slabs> <num_slabs> <sharedavail>
rpc_buffers 8 8 2048 2 1 : tunables 24 12
0 : slabdata 4 4 0
rpc_tasks 8 24 160 24 1 : tunables 120 60
0 : slabdata 1 1 0
rpc_inode_cache 0 0 416 9 1 : tunables 54 27
0 : slabdata 0 0 0
flow_cache 0 0 80 48 1 : tunables 120 60
0 : slabdata 0 0 0
nfs_direct_cache 0 0 68 56 1 : tunables 120 60
0 : slabdata 0 0 0
nfs_write_data 36 36 416 9 1 : tunables 54 27
0 : slabdata 4 4 0
nfs_read_data 32 36 416 9 1 : tunables 54 27
0 : slabdata 4 4 0
nfs_inode_cache 0 0 584 7 1 : tunables 54 27
0 : slabdata 0 0 0
nfs_page 0 0 64 59 1 : tunables 120 60
0 : slabdata 0 0 0
journal_handle 0 0 20 169 1 : tunables 120 60
0 : slabdata 0 0 0
journal_head 14 126 60 63 1 : tunables 120 60
0 : slabdata 2 2 0
revoke_table 2 254 12 254 1 : tunables 120 60
0 : slabdata 1 1 0
revoke_record 0 0 16 203 1 : tunables 120 60
0 : slabdata 0 0 0
ext2_inode_cache 0 0 416 9 1 : tunables 54 27
0 : slabdata 0 0 0
ext3_inode_cache 443 450 432 9 1 : tunables 54 27
0 : slabdata 50 50 0
ext3_xattr 0 0 44 84 1 : tunables 120 60
0 : slabdata 0 0 0
reiser_inode_cache 0 0 368 10 1 : tunables 54 27
0 : slabdata 0 0 0
configfs_dir_cache 0 0 52 72 1 : tunables 120 60
0 : slabdata 0 0 0
kioctx 0 0 160 24 1 : tunables 120 60
0 : slabdata 0 0 0
kiocb 0 0 160 24 1 : tunables 120 60
0 : slabdata 0 0 0
inotify_event_cache 0 0 28 127 1 : tunables 120 60
0 : slabdata 0 0 0
inotify_watch_cache 2 92 40 92 1 : tunables 120 60
0 : slabdata 1 1 0
dnotify_cache 0 0 20 169 1 : tunables 120 60
0 : slabdata 0 0 0
fasync_cache 0 0 16 203 1 : tunables 120 60
0 : slabdata 0 0 0
shmem_inode_cache 2497 2500 392 10 1 : tunables 54 27
0 : slabdata 250 250 0
nsproxy 0 0 24 145 1 : tunables 120 60
0 : slabdata 0 0 0
posix_timers_cache 0 0 112 35 1 : tunables 120 60
0 : slabdata 0 0 0
uid_cache 0 0 64 59 1 : tunables 120 60
0 : slabdata 0 0 0
UNIX 5 10 384 10 1 : tunables 54 27
0 : slabdata 1 1 0
UDP-Lite 0 0 480 8 1 : tunables 54 27
0 : slabdata 0 0 0
tcp_bind_bucket 1 113 32 113 1 : tunables 120 60
0 : slabdata 1 1 0
inet_peer_cache 0 0 64 59 1 : tunables 120 60
0 : slabdata 0 0 0
secpath_cache 0 0 32 113 1 : tunables 120 60
0 : slabdata 0 0 0
xfrm_dst_cache 0 0 288 13 1 : tunables 54 27
0 : slabdata 0 0 0
ip_fib_alias 0 0 16 203 1 : tunables 120 60
0 : slabdata 0 0 0
ip_fib_hash 9 101 36 101 1 : tunables 120 60
0 : slabdata 1 1 0
ip_dst_cache 388 435 256 15 1 : tunables 120 60
0 : slabdata 29 29 0
arp_cache 1 30 128 30 1 : tunables 120 60
0 : slabdata 1 1 0
RAW 2 9 448 9 1 : tunables 54 27
0 : slabdata 1 1 0
UDP 1 8 480 8 1 : tunables 54 27
0 : slabdata 1 1 0
tw_sock_TCP 0 0 96 40 1 : tunables 120 60
0 : slabdata 0 0 0
request_sock_TCP 0 0 96 40 1 : tunables 120 60
0 : slabdata 0 0 0
TCP 2 3 1056 3 1 : tunables 24 12
0 : slabdata 1 1 0
eventpoll_pwq 0 0 36 101 1 : tunables 120 60
0 : slabdata 0 0 0
eventpoll_epi 0 0 96 40 1 : tunables 120 60
0 : slabdata 0 0 0
sgpool-128 2 2 2048 2 1 : tunables 24 12
0 : slabdata 1 1 0
sgpool-64 2 4 1024 4 1 : tunables 54 27
0 : slabdata 1 1 0
sgpool-32 2 8 512 8 1 : tunables 54 27
0 : slabdata 1 1 0
sgpool-16 2 15 256 15 1 : tunables 120 60
0 : slabdata 1 1 0
sgpool-8 2 30 128 30 1 : tunables 120 60
0 : slabdata 1 1 0
scsi_data_buffer 0 0 20 169 1 : tunables 120 60
0 : slabdata 0 0 0
blkdev_queue 10 12 1216 3 1 : tunables 24 12
0 : slabdata 4 4 0
blkdev_requests 8 18 216 18 1 : tunables 120 60
0 : slabdata 1 1 0
blkdev_ioc 10 84 44 84 1 : tunables 120 60
0 : slabdata 1 1 0
bio-0 2 30 128 30 1 : tunables 120 60
0 : slabdata 1 1 0
biovec-256 2 2 3072 1 1 : tunables 24 12
0 : slabdata 2 2 0
biovec-128 0 0 1536 2 1 : tunables 24 12
0 : slabdata 0 0 0
biovec-64 0 0 768 5 1 : tunables 54 27
0 : slabdata 0 0 0
biovec-16 0 0 192 20 1 : tunables 120 60
0 : slabdata 0 0 0
sock_inode_cache 18 22 352 11 1 : tunables 54 27
0 : slabdata 2 2 0
skbuff_fclone_cache 11 11 352 11 1 : tunables 54 27
0 : slabdata 1 1 0
skbuff_head_cache 100 180 192 20 1 : tunables 120 60
0 : slabdata 9 9 0
file_lock_cache 1 40 96 40 1 : tunables 120 60
0 : slabdata 1 1 0
proc_inode_cache 132 132 320 12 1 : tunables 54 27
0 : slabdata 11 11 0
sigqueue 1 27 144 27 1 : tunables 120 60
0 : slabdata 1 1 0
radix_tree_node 289 299 288 13 1 : tunables 54 27
0 : slabdata 23 23 0
bdev_cache 3 9 416 9 1 : tunables 54 27
0 : slabdata 1 1 0
sysfs_dir_cache 3902 3948 44 84 1 : tunables 120 60
0 : slabdata 47 47 0
mnt_cache 20 30 128 30 1 : tunables 120 60
0 : slabdata 1 1 0
filp 210 210 128 30 1 : tunables 120 60
0 : slabdata 7 7 0
inode_cache 1560 1560 296 13 1 : tunables 54 27
0 : slabdata 120 120 0
dentry 4829 4830 128 30 1 : tunables 120 60
0 : slabdata 161 161 0
names_cache 1 1 4096 1 1 : tunables 24 12
0 : slabdata 1 1 0
buffer_head 614 648 52 72 1 : tunables 120 60
0 : slabdata 9 9 0
vm_area_struct 631 644 84 46 1 : tunables 120 60
0 : slabdata 14 14 0
mm_struct 20 20 384 10 1 : tunables 54 27
0 : slabdata 2 2 0
fs_cache 10 113 32 113 1 : tunables 120 60
0 : slabdata 1 1 0
files_cache 11 40 192 20 1 : tunables 120 60
0 : slabdata 2 2 0
signal_cache 45 45 448 9 1 : tunables 54 27
0 : slabdata 5 5 0
sighand_cache 36 36 1312 3 1 : tunables 24 12
0 : slabdata 12 12 0
task_struct 40 40 768 5 1 : tunables 54 27
0 : slabdata 8 8 0
cred_jar 56 120 96 40 1 : tunables 120 60
0 : slabdata 3 3 0
anon_vma 265 339 8 339 1 : tunables 120 60
0 : slabdata 1 1 0
pid 35 59 64 59 1 : tunables 120 60
0 : slabdata 1 1 0
idr_layer_cache 127 130 148 26 1 : tunables 120 60
0 : slabdata 5 5 0
size-4194304 0 0 4194304 1 1024 : tunables 1 1
0 : slabdata 0 0 0
size-2097152 0 0 2097152 1 512 : tunables 1 1
0 : slabdata 0 0 0
size-1048576 0 0 1048576 1 256 : tunables 1 1
0 : slabdata 0 0 0
size-524288 0 0 524288 1 128 : tunables 1 1
0 : slabdata 0 0 0
size-262144 0 0 262144 1 64 : tunables 1 1
0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4
0 : slabdata 0 0 0
size-65536 0 0 65536 1 16 : tunables 8 4
0 : slabdata 0 0 0
size-32768 0 0 32768 1 8 : tunables 8 4
0 : slabdata 0 0 0
size-16384 0 0 16384 1 4 : tunables 8 4
0 : slabdata 0 0 0
size-8192 0 0 8192 1 2 : tunables 8 4
0 : slabdata 0 0 0
size-4096 4 4 4096 1 1 : tunables 24 12
0 : slabdata 4 4 0
size-2048 14 14 2048 2 1 : tunables 24 12
0 : slabdata 7 7 0
size-1024 40 40 1024 4 1 : tunables 54 27
0 : slabdata 10 10 0
size-512 173 200 512 8 1 : tunables 54 27
0 : slabdata 25 25 0
size-256 75 75 256 15 1 : tunables 120 60
0 : slabdata 5 5 0
size-192 669 680 192 20 1 : tunables 120 60
0 : slabdata 34 34 0
size-128 240 240 128 30 1 : tunables 120 60
0 : slabdata 8 8 0
size-96 919 920 96 40 1 : tunables 120 60
0 : slabdata 23 23 0
size-64 590 590 64 59 1 : tunables 120 60
0 : slabdata 10 10 0
size-32 3374 3390 32 113 1 : tunables 120 60
0 : slabdata 30 30 0
kmem_cache 105 120 96 40 1 : tunables 120 60
0 : slabdata 3 3 0
> ...
debian:~# dmesg
Linux version 2.6.30-00002-g0148992 (kconstan@...ian) (gcc version 4.3.2
(Debian 4.3.2-1.1) ) #16 PREEMPT Fri Dec 11 13:48:45 PST 2009
CPU: ARM920T [41129200] revision 0 (ARMv4T), cr=c0007177
CPU: VIVT data cache, VIVT instruction cache
Machine: emQbit's ECB_AT91
Memory policy: ECC disabled, Data cache writeback
On node 0 totalpages: 8192
free_area_init_node: node 0, pgdat c03a3340, node_mem_map c03c3000
Normal zone: 64 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 8128 pages, LIFO batch:0
Clocks: CPU 179 MHz, master 59 MHz, main 18.432 MHz
Built 1 zonelists in Zone order, mobility grouping on. Total pages:
8128
Kernel command line: mem=32M root=/dev/mmcblk0p1 ip=192.168.0.51
console=ttyS0,115200n8 rootdelay=4
NR_IRQS:192
AT91: 96 gpio irqs in 3 banks
...
eth0: Link now 100-FullDuplex
eth0: AT91 ethernet at 0xfefbc000 int=24 100-FullDuplex (00:00:00:00:00:5b)
eth0: Micrel KS8721 PHY
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
-kevin
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists