| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1445961371.6332.36.camel@linux.intel.com>
Date: Tue, 27 Oct 2015 17:56:11 +0200
From: Andy Shevchenko <andriy.shevchenko@...ux.intel.com>
To: Aaro Koskinen <aaro.koskinen@...ia.com>,
Catalin Marinas <catalin.marinas@....com>,
Andrew Morton <akpm@...ux-foundation.org>,
Linus Torvalds <torvalds@...ux-foundation.org>,
linux-kernel@...r.kernel.org
Subject: Re: 4.3-rc7: kmemleak BUG: Redzone overwritten
On Tue, 2015-10-27 at 17:46 +0200, Aaro Koskinen wrote:
> Hi,
>
> With 4.3-rc7 and slub_debug=FZUP, I get the below when reading
> /sys/kernel/debug/kmemleak with a large number of reported entries.
> It's pretty repeatable. HW is MIPS64.
>
> With the SLUB debugging disabled, box crashes randomly in
> kmem_cache_free
> or kmem_cache_alloc when the kmemleak file is read on a running
> system.
>
> Seems to start with 6fc37c490076 ("kmemleak: use seq_hex_dump() to
> dump buffers").
So, you mean reverting it does help?
Btw, we have kmemleak test suite. Any suggestion how can it be
reproducible with it?
>
> A.
>
> ---8<---
>
> [ 77.706850]
> =====================================================================
> ========
> [ 77.706871] BUG kmalloc-4096 (Not tainted): Redzone overwritten
> [ 77.706877] ------------------------------------------------------
> -----------------------
> [ 77.706877]
> [ 77.706885] Disabling lock debugging due to kernel taint
> [ 77.706894] INFO: 0x800000002e939000-0x800000002e939000. First
> byte 0x0 instead of 0xcc
> [ 77.706914] INFO: Allocated in seq_buf_alloc+0x24/0x58 age=452
> cpu=2 pid=587
> [ 77.706928] __slab_alloc.isra.72.constprop.75+0x4a4/0x508
> [ 77.706938] __kmalloc+0x30c/0x3f0
> [ 77.706947] seq_buf_alloc+0x24/0x58
> [ 77.706956] seq_read+0x304/0x4a0
> [ 77.706968] __vfs_read+0x3c/0x100
> [ 77.706977] vfs_read+0x8c/0x138
> [ 77.706987] SyS_read+0x64/0xe8
> [ 77.707000] syscall_common+0x34/0x58
> [ 77.707012] INFO: Freed in seq_release+0x24/0x40 age=3450 cpu=3
> pid=584
> [ 77.707023] __slab_free+0x340/0x4f0
> [ 77.707032] seq_release+0x24/0x40
> [ 77.707044] kernfs_fop_release+0x50/0x80
> [ 77.707055] __fput+0xa4/0x218
> [ 77.707066] task_work_run+0xb0/0x108
> [ 77.707078] work_notifysig+0x10/0x18
> [ 77.707087] INFO: Slab 0x8000000003ec4440 objects=7 used=1
> fp=0x800000002e93e7b0 flags=0x200000004081
> [ 77.707095] INFO: Object 0x800000002e938000 @offset=0
> fp=0x800000002e939148
> [ 77.707095]
> [ 77.707108] Object 800000002e938000: 75 6e 72 65 66 65 72 65 6e 63
> 65 64 20 6f 62 6a unreferenced obj
> [ 77.707118] Object 800000002e938010: 65 63 74 20 30 78 38 30 30 30
> 30 30 30 30 32 66 ect 0x800000002f
> [ 77.707128] Object 800000002e938020: 33 37 32 65 36 38 20 28 73 69
> 7a 65 20 32 30 34 372e68 (size 204
> [ 77.707138] Object 800000002e938030: 38 29 3a 0a 20 20 63 6f 6d 6d
> 20 22 73 77 61 70 8):. comm "swap
> [ 77.707148] Object 800000002e938040: 70 65 72 2f 30 22 2c 20 70 69
> 64 20 31 2c 20 6a per/0", pid 1, j
> [ 77.707158] Object 800000002e938050: 69 66 66 69 65 73 20 34 32 39
> 34 39 33 38 30 35 iffies 429493805
> [ 77.707168] Object 800000002e938060: 31 20 28 61 67 65 20 34 31 2e
> 35 37 30 73 29 0a 1 (age 41.570s).
> [ 77.707177] Object 800000002e938070: 20 20 68 65 78 20 64 75 6d 70
> 20 28 66 69 72 73 hex dump (firs
> [ 77.707188] Object 800000002e938080: 74 20 33 32 20 62 79 74 65 73
> 29 3a 0a 20 20 20 t 32 bytes):.
> [ 77.707197] Object 800000002e938090: 20 36 62 20 36 62 20 36 62 20
> 36 62 20 36 62 20 6b 6b 6b 6b 6b
> [ 77.707207] Object 800000002e9380a0: 36 62 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6b 6
> [ 77.707217] Object 800000002e9380b0: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.707227] Object 800000002e9380c0: 20 20 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkk
> [ 77.707237] Object 800000002e9380d0: 6b 6b 0a 20 20 20 20 36 62 20
> 36 62 20 36 62 20 kk. 6b 6b 6b
> [ 77.707247] Object 800000002e9380e0: 36 62 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6b 6
> [ 77.707257] Object 800000002e9380f0: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.707267] Object 800000002e938100: 20 36 62 20 36 62 20 20 6b 6b
> 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkk
> [ 77.707277] Object 800000002e938110: 6b 6b 6b 6b 6b 6b 6b 6b 0a 20
> 20 62 61 63 6b 74 kkkkkkkk. backt
> [ 77.707287] Object 800000002e938120: 72 61 63 65 3a 0a 20 20 20 20
> 5b 3c 66 66 66 66 race:. [<ffff
> [ 77.707297] Object 800000002e938130: 66 66 66 66 38 31 32 36 63 39
> 62 38 3e 5d 20 5f ffff8126c9b8>] _
> [ 77.707307] Object 800000002e938140: 5f 6b 6d 61 6c 6c 6f 63 2b 30
> 78 31 62 38 2f 30 _kmalloc+0x1b8/0
> [ 77.707317] Object 800000002e938150: 78 33 66 30 0a 20 20 20 20 5b
> 3c 66 66 66 66 66 x3f0. [<fffff
> [ 77.707327] Object 800000002e938160: 66 66 66 38 31 34 65 31 31 62
> 38 3e 5d 20 63 76 fff814e11b8>] cv
> [ 77.707337] Object 800000002e938170: 6d 5f 6f 63 74 5f 6d 65 6d 5f
> 66 69 6c 6c 5f 66 m_oct_mem_fill_f
> [ 77.707347] Object 800000002e938180: 70 61 2b 30 78 37 38 2f 30 78
> 31 64 38 0a 20 20 pa+0x78/0x1d8.
> [ 77.707357] Object 800000002e938190: 20 20 5b 3c 66 66 66 66 66 66
> 66 66 38 31 34 64 [<ffffffff814d
> [ 77.707367] Object 800000002e9381a0: 66 64 33 38 3e 5d 20 63 76 6d
> 5f 6f 63 74 5f 70 fd38>] cvm_oct_p
> [ 77.707377] Object 800000002e9381b0: 72 6f 62 65 2b 30 78 66 38 2f
> 30 78 37 62 30 0a robe+0xf8/0x7b0.
> [ 77.707386] Object 800000002e9381c0: 20 20 20 20 5b 3c 66 66 66 66
> 66 66 66 66 38 31 [<ffffffff81
> [ 77.707396] Object 800000002e9381d0: 34 35 32 37 61 38 3e 5d 20 70
> 6c 61 74 66 6f 72 4527a8>] platfor
> [ 77.707406] Object 800000002e9381e0: 6d 5f 64 72 76 5f 70 72 6f 62
> 65 2b 30 78 34 38 m_drv_probe+0x48
> [ 77.707416] Object 800000002e9381f0: 2f 30 78 64 38 0a 20 20 20 20
> 5b 3c 66 66 66 66 /0xd8. [<ffff
> [ 77.707426] Object 800000002e938200: 66 66 66 66 38 31 34 35 30 31
> 34 63 3e 5d 20 64 ffff8145014c>] d
> [ 77.707436] Object 800000002e938210: 72 69 76 65 72 5f 70 72 6f 62
> 65 5f 64 65 76 69 river_probe_devi
> [ 77.707446] Object 800000002e938220: 63 65 2b 30 78 32 39 63 2f 30
> 78 33 35 30 0a 20 ce+0x29c/0x350.
> [ 77.707456] Object 800000002e938230: 20 20 20 5b 3c 66 66 66 66 66
> 66 66 66 38 31 34 [<ffffffff814
> [ 77.707466] Object 800000002e938240: 35 30 32 64 30 3e 5d 20 5f 5f
> 64 72 69 76 65 72 502d0>] __driver
> [ 77.707476] Object 800000002e938250: 5f 61 74 74 61 63 68 2b 30 78
> 64 30 2f 30 78 64 _attach+0xd0/0xd
> [ 77.707486] Object 800000002e938260: 38 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 8. [<ffffffff
> [ 77.707496] Object 800000002e938270: 38 31 34 34 64 63 32 34 3e 5d
> 20 62 75 73 5f 66 8144dc24>] bus_f
> [ 77.707506] Object 800000002e938280: 6f 72 5f 65 61 63 68 5f 64 65
> 76 2b 30 78 37 34 or_each_dev+0x74
> [ 77.707516] Object 800000002e938290: 2f 30 78 63 30 0a 20 20 20 20
> 5b 3c 66 66 66 66 /0xc0. [<ffff
> [ 77.707526] Object 800000002e9382a0: 66 66 66 66 38 31 34 34 66 35
> 39 38 3e 5d 20 62 ffff8144f598>] b
> [ 77.707536] Object 800000002e9382b0: 75 73 5f 61 64 64 5f 64 72 69
> 76 65 72 2b 30 78 us_add_driver+0x
> [ 77.707545] Object 800000002e9382c0: 32 30 38 2f 30 78 32 38 30 0a
> 20 20 20 20 5b 3c 208/0x280. [<
> [ 77.707555] Object 800000002e9382d0: 66 66 66 66 66 66 66 66 38 31
> 34 35 31 30 30 30 ffffffff81451000
> [ 77.707565] Object 800000002e9382e0: 3e 5d 20 64 72 69 76 65 72 5f
> 72 65 67 69 73 74 >] driver_regist
> [ 77.707575] Object 800000002e9382f0: 65 72 2b 30 78 39 30 2f 30 78
> 31 33 38 0a 20 20 er+0x90/0x138.
> [ 77.707585] Object 800000002e938300: 20 20 5b 3c 66 66 66 66 66 66
> 66 66 38 31 31 30 [<ffffffff8110
> [ 77.707595] Object 800000002e938310: 30 35 35 30 3e 5d 20 64 6f 5f
> 6f 6e 65 5f 69 6e 0550>] do_one_in
> [ 77.707605] Object 800000002e938320: 69 74 63 61 6c 6c 2b 30 78 61
> 30 2f 30 78 31 63 itcall+0xa0/0x1c
> [ 77.707615] Object 800000002e938330: 30 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 0. [<ffffffff
> [ 77.707625] Object 800000002e938340: 38 31 38 33 63 65 31 34 3e 5d
> 20 6b 65 72 6e 65 8183ce14>] kerne
> [ 77.707635] Object 800000002e938350: 6c 5f 69 6e 69 74 5f 66 72 65
> 65 61 62 6c 65 2b l_init_freeable+
> [ 77.707645] Object 800000002e938360: 30 78 31 38 38 2f 30 78 32 34
> 34 0a 20 20 20 20 0x188/0x244.
> [ 77.707655] Object 800000002e938370: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 36 31 61 30 [<ffffffff8161a0
> [ 77.707665] Object 800000002e938380: 31 38 3e 5d 20 6b 65 72 6e 65
> 6c 5f 69 6e 69 74 18>] kernel_init
> [ 77.707675] Object 800000002e938390: 2b 30 78 32 30 2f 30 78 31 31
> 38 0a 20 20 20 20 +0x20/0x118.
> [ 77.707685] Object 800000002e9383a0: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 31 32 30 61 [<ffffffff81120a
> [ 77.707695] Object 800000002e9383b0: 36 63 3e 5d 20 72 65 74 5f 66
> 72 6f 6d 5f 6b 65 6c>] ret_from_ke
> [ 77.707705] Object 800000002e9383c0: 72 6e 65 6c 5f 74 68 72 65 61
> 64 2b 30 78 31 34 rnel_thread+0x14
> [ 77.707714] Object 800000002e9383d0: 2f 30 78 31 63 0a 00 6e 72 65
> 66 65 72 65 6e 63 /0x1c..nreferenc
> [ 77.707724] Object 800000002e9383e0: 65 64 20 6f 62 6a 65 63 74 20
> 30 78 38 30 30 30 ed object 0x8000
> [ 77.707734] Object 800000002e9383f0: 30 30 30 30 32 66 33 37 35 33
> 38 38 20 28 73 69 00002f375388 (si
> [ 77.707744] Object 800000002e938400: 7a 65 20 32 30 34 38 29 3a 0a
> 20 20 63 6f 6d 6d ze 2048):. comm
> [ 77.707754] Object 800000002e938410: 20 22 73 77 61 70 70 65 72 2f
> 30 22 2c 20 70 69 "swapper/0", pi
> [ 77.707764] Object 800000002e938420: 64 20 31 2c 20 6a 69 66 66 69
> 65 73 20 34 32 39 d 1, jiffies 429
> [ 77.707774] Object 800000002e938430: 34 39 33 38 30 35 31 20 28 61
> 67 65 20 34 31 2e 4938051 (age 41.
> [ 77.707784] Object 800000002e938440: 35 37 30 73 29 0a 20 20 68 65
> 78 20 64 75 6d 70 570s). hex dump
> [ 77.707794] Object 800000002e938450: 20 28 66 69 72 73 74 20 33 32
> 20 62 79 74 65 73 (first 32 bytes
> [ 77.707804] Object 800000002e938460: 29 3a 0a 20 20 20 20 36 62 20
> 36 62 20 36 62 20 ):. 6b 6b 6b
> [ 77.707814] Object 800000002e938470: 36 62 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6b 6
> [ 77.707824] Object 800000002e938480: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.707834] Object 800000002e938490: 20 36 62 20 36 62 20 20 6b 6b
> 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkk
> [ 77.707844] Object 800000002e9384a0: 6b 6b 6b 6b 6b 6b 6b 6b 0a 20
> 20 20 20 36 62 20 kkkkkkkk. 6b
> [ 77.707854] Object 800000002e9384b0: 36 62 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6b 6
> [ 77.707864] Object 800000002e9384c0: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.707873] Object 800000002e9384d0: 20 36 62 20 36 62 20 36 62 20
> 36 62 20 20 6b 6b 6b 6b 6b 6b kk
> [ 77.707883] Object 800000002e9384e0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b 6b 0a 20 kkkkkkkkkkkkkk.
> [ 77.707893] Object 800000002e9384f0: 20 62 61 63 6b 74 72 61 63 65
> 3a 0a 20 20 20 20 backtrace:.
> [ 77.707903] Object 800000002e938500: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 32 36 63 39 [<ffffffff8126c9
> [ 77.707913] Object 800000002e938510: 62 38 3e 5d 20 5f 5f 6b 6d 61
> 6c 6c 6f 63 2b 30 b8>] __kmalloc+0
> [ 77.707923] Object 800000002e938520: 78 31 62 38 2f 30 78 33 66 30
> 0a 20 20 20 20 5b x1b8/0x3f0. [
> [ 77.707933] Object 800000002e938530: 3c 66 66 66 66 66 66 66 66 38
> 31 34 65 31 31 62 <ffffffff814e11b
> [ 77.707943] Object 800000002e938540: 38 3e 5d 20 63 76 6d 5f 6f 63
> 74 5f 6d 65 6d 5f 8>] cvm_oct_mem_
> [ 77.707953] Object 800000002e938550: 66 69 6c 6c 5f 66 70 61 2b 30
> 78 37 38 2f 30 78 fill_fpa+0x78/0x
> [ 77.707963] Object 800000002e938560: 31 64 38 0a 20 20 20 20 5b 3c
> 66 66 66 66 66 66 1d8. [<ffffff
> [ 77.707973] Object 800000002e938570: 66 66 38 31 34 64 66 64 33 38
> 3e 5d 20 63 76 6d ff814dfd38>] cvm
> [ 77.707983] Object 800000002e938580: 5f 6f 63 74 5f 70 72 6f 62 65
> 2b 30 78 66 38 2f _oct_probe+0xf8/
> [ 77.707993] Object 800000002e938590: 30 78 37 62 30 0a 20 20 20 20
> 5b 3c 66 66 66 66 0x7b0. [<ffff
> [ 77.708003] Object 800000002e9385a0: 66 66 66 66 38 31 34 35 32 37
> 61 38 3e 5d 20 70 ffff814527a8>] p
> [ 77.708013] Object 800000002e9385b0: 6c 61 74 66 6f 72 6d 5f 64 72
> 76 5f 70 72 6f 62 latform_drv_prob
> [ 77.708023] Object 800000002e9385c0: 65 2b 30 78 34 38 2f 30 78 64
> 38 0a 20 20 20 20 e+0x48/0xd8.
> [ 77.708033] Object 800000002e9385d0: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 34 35 30 31 [<ffffffff814501
> [ 77.708042] Object 800000002e9385e0: 34 63 3e 5d 20 64 72 69 76 65
> 72 5f 70 72 6f 62 4c>] driver_prob
> [ 77.708052] Object 800000002e9385f0: 65 5f 64 65 76 69 63 65 2b 30
> 78 32 39 63 2f 30 e_device+0x29c/0
> [ 77.708062] Object 800000002e938600: 78 33 35 30 0a 20 20 20 20 5b
> 3c 66 66 66 66 66 x350. [<fffff
> [ 77.708072] Object 800000002e938610: 66 66 66 38 31 34 35 30 32 64
> 30 3e 5d 20 5f 5f fff814502d0>] __
> [ 77.708082] Object 800000002e938620: 64 72 69 76 65 72 5f 61 74 74
> 61 63 68 2b 30 78 driver_attach+0x
> [ 77.708092] Object 800000002e938630: 64 30 2f 30 78 64 38 0a 20 20
> 20 20 5b 3c 66 66 d0/0xd8. [<ff
> [ 77.708102] Object 800000002e938640: 66 66 66 66 66 66 38 31 34 34
> 64 63 32 34 3e 5d ffffff8144dc24>]
> [ 77.708112] Object 800000002e938650: 20 62 75 73 5f 66 6f 72 5f 65
> 61 63 68 5f 64 65 bus_for_each_de
> [ 77.708122] Object 800000002e938660: 76 2b 30 78 37 34 2f 30 78 63
> 30 0a 20 20 20 20 v+0x74/0xc0.
> [ 77.708132] Object 800000002e938670: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 34 34 66 35 [<ffffffff8144f5
> [ 77.708142] Object 800000002e938680: 39 38 3e 5d 20 62 75 73 5f 61
> 64 64 5f 64 72 69 98>] bus_add_dri
> [ 77.708152] Object 800000002e938690: 76 65 72 2b 30 78 32 30 38 2f
> 30 78 32 38 30 0a ver+0x208/0x280.
> [ 77.708162] Object 800000002e9386a0: 20 20 20 20 5b 3c 66 66 66 66
> 66 66 66 66 38 31 [<ffffffff81
> [ 77.708172] Object 800000002e9386b0: 34 35 31 30 30 30 3e 5d 20 64
> 72 69 76 65 72 5f 451000>] driver_
> [ 77.708182] Object 800000002e9386c0: 72 65 67 69 73 74 65 72 2b 30
> 78 39 30 2f 30 78 register+0x90/0x
> [ 77.708192] Object 800000002e9386d0: 31 33 38 0a 20 20 20 20 5b 3c
> 66 66 66 66 66 66 138. [<ffffff
> [ 77.708202] Object 800000002e9386e0: 66 66 38 31 31 30 30 35 35 30
> 3e 5d 20 64 6f 5f ff81100550>] do_
> [ 77.708211] Object 800000002e9386f0: 6f 6e 65 5f 69 6e 69 74 63 61
> 6c 6c 2b 30 78 61 one_initcall+0xa
> [ 77.708221] Object 800000002e938700: 30 2f 30 78 31 63 30 0a 20 20
> 20 20 5b 3c 66 66 0/0x1c0. [<ff
> [ 77.708231] Object 800000002e938710: 66 66 66 66 66 66 38 31 38 33
> 63 65 31 34 3e 5d ffffff8183ce14>]
> [ 77.708241] Object 800000002e938720: 20 6b 65 72 6e 65 6c 5f 69 6e
> 69 74 5f 66 72 65 kernel_init_fre
> [ 77.708251] Object 800000002e938730: 65 61 62 6c 65 2b 30 78 31 38
> 38 2f 30 78 32 34 eable+0x188/0x24
> [ 77.708261] Object 800000002e938740: 34 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 4. [<ffffffff
> [ 77.708271] Object 800000002e938750: 38 31 36 31 61 30 31 38 3e 5d
> 20 6b 65 72 6e 65 8161a018>] kerne
> [ 77.708281] Object 800000002e938760: 6c 5f 69 6e 69 74 2b 30 78 32
> 30 2f 30 78 31 31 l_init+0x20/0x11
> [ 77.708291] Object 800000002e938770: 38 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 8. [<ffffffff
> [ 77.708301] Object 800000002e938780: 38 31 31 32 30 61 36 63 3e 5d
> 20 72 65 74 5f 66 81120a6c>] ret_f
> [ 77.708311] Object 800000002e938790: 72 6f 6d 5f 6b 65 72 6e 65 6c
> 5f 74 68 72 65 61 rom_kernel_threa
> [ 77.708321] Object 800000002e9387a0: 64 2b 30 78 31 34 2f 30 78 31
> 63 0a 75 6e 72 65 d+0x14/0x1c.unre
> [ 77.708331] Object 800000002e9387b0: 66 65 72 65 6e 63 65 64 20 6f
> 62 6a 65 63 74 20 ferenced object
> [ 77.708341] Object 800000002e9387c0: 30 78 38 30 30 30 30 30 30 30
> 32 66 33 37 32 35 0x800000002f3725
> [ 77.708351] Object 800000002e9387d0: 32 30 20 28 73 69 7a 65 20 32
> 30 34 38 29 3a 0a 20 (size 2048):.
> [ 77.708361] Object 800000002e9387e0: 20 20 63 6f 6d 6d 20 22 73 77
> 61 70 70 65 72 2f comm "swapper/
> [ 77.708370] Object 800000002e9387f0: 30 22 2c 20 70 69 64 20 31 2c
> 20 6a 69 66 66 69 0", pid 1, jiffi
> [ 77.708381] Object 800000002e938800: 65 73 20 34 32 39 34 39 33 38
> 30 35 31 20 28 61 es 4294938051 (a
> [ 77.708390] Object 800000002e938810: 67 65 20 34 31 2e 35 37 30 73
> 29 0a 20 20 68 65 ge 41.570s). he
> [ 77.708400] Object 800000002e938820: 78 20 64 75 6d 70 20 28 66 69
> 72 73 74 20 33 32 x dump (first 32
> [ 77.708410] Object 800000002e938830: 20 62 79 74 65 73 29 3a 0a 20
> 20 20 20 36 62 20 bytes):. 6b
> [ 77.708420] Object 800000002e938840: 36 62 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6b 6
> [ 77.708430] Object 800000002e938850: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.708440] Object 800000002e938860: 20 36 62 20 36 62 20 36 62 20
> 36 62 20 20 6b 6b 6b 6b 6b 6b kk
> [ 77.708450] Object 800000002e938870: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 6b 6b 6b 6b 0a 20 kkkkkkkkkkkkkk.
> [ 77.708460] Object 800000002e938880: 20 20 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6
> [ 77.708470] Object 800000002e938890: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.708480] Object 800000002e9388a0: 20 36 62 20 36 62 20 36 62 20
> 36 62 20 36 62 20 6b 6b 6b 6b 6b
> [ 77.708490] Object 800000002e9388b0: 36 62 20 20 6b 6b 6b 6b 6b 6b
> 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkk
> [ 77.708500] Object 800000002e9388c0: 6b 6b 6b 6b 0a 20 20 62 61 63
> 6b 74 72 61 63 65 kkkk. backtrace
> [ 77.708510] Object 800000002e9388d0: 3a 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 :. [<ffffffff
> [ 77.708520] Object 800000002e9388e0: 38 31 32 36 63 39 62 38 3e 5d
> 20 5f 5f 6b 6d 61 8126c9b8>] __kma
> [ 77.708530] Object 800000002e9388f0: 6c 6c 6f 63 2b 30 78 31 62 38
> 2f 30 78 33 66 30 lloc+0x1b8/0x3f0
> [ 77.708540] Object 800000002e938900: 0a 20 20 20 20 5b 3c 66 66 66
> 66 66 66 66 66 38 . [<ffffffff8
> [ 77.708549] Object 800000002e938910: 31 34 65 31 31 62 38 3e 5d 20
> 63 76 6d 5f 6f 63 14e11b8>] cvm_oc
> [ 77.708559] Object 800000002e938920: 74 5f 6d 65 6d 5f 66 69 6c 6c
> 5f 66 70 61 2b 30 t_mem_fill_fpa+0
> [ 77.708569] Object 800000002e938930: 78 37 38 2f 30 78 31 64 38 0a
> 20 20 20 20 5b 3c x78/0x1d8. [<
> [ 77.708579] Object 800000002e938940: 66 66 66 66 66 66 66 66 38 31
> 34 64 66 64 33 38 ffffffff814dfd38
> [ 77.708589] Object 800000002e938950: 3e 5d 20 63 76 6d 5f 6f 63 74
> 5f 70 72 6f 62 65 >] cvm_oct_probe
> [ 77.708599] Object 800000002e938960: 2b 30 78 66 38 2f 30 78 37 62
> 30 0a 20 20 20 20 +0xf8/0x7b0.
> [ 77.708609] Object 800000002e938970: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 34 35 32 37 [<ffffffff814527
> [ 77.708619] Object 800000002e938980: 61 38 3e 5d 20 70 6c 61 74 66
> 6f 72 6d 5f 64 72 a8>] platform_dr
> [ 77.708629] Object 800000002e938990: 76 5f 70 72 6f 62 65 2b 30 78
> 34 38 2f 30 78 64 v_probe+0x48/0xd
> [ 77.708639] Object 800000002e9389a0: 38 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 8. [<ffffffff
> [ 77.708649] Object 800000002e9389b0: 38 31 34 35 30 31 34 63 3e 5d
> 20 64 72 69 76 65 8145014c>] drive
> [ 77.708659] Object 800000002e9389c0: 72 5f 70 72 6f 62 65 5f 64 65
> 76 69 63 65 2b 30 r_probe_device+0
> [ 77.708669] Object 800000002e9389d0: 78 32 39 63 2f 30 78 33 35 30
> 0a 20 20 20 20 5b x29c/0x350. [
> [ 77.708679] Object 800000002e9389e0: 3c 66 66 66 66 66 66 66 66 38
> 31 34 35 30 32 64 <ffffffff814502d
> [ 77.708689] Object 800000002e9389f0: 30 3e 5d 20 5f 5f 64 72 69 76
> 65 72 5f 61 74 74 0>] __driver_att
> [ 77.708699] Object 800000002e938a00: 61 63 68 2b 30 78 64 30 2f 30
> 78 64 38 0a 20 20 ach+0xd0/0xd8.
> [ 77.708709] Object 800000002e938a10: 20 20 5b 3c 66 66 66 66 66 66
> 66 66 38 31 34 34 [<ffffffff8144
> [ 77.708718] Object 800000002e938a20: 64 63 32 34 3e 5d 20 62 75 73
> 5f 66 6f 72 5f 65 dc24>] bus_for_e
> [ 77.708728] Object 800000002e938a30: 61 63 68 5f 64 65 76 2b 30 78
> 37 34 2f 30 78 63 ach_dev+0x74/0xc
> [ 77.708738] Object 800000002e938a40: 30 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 0. [<ffffffff
> [ 77.708748] Object 800000002e938a50: 38 31 34 34 66 35 39 38 3e 5d
> 20 62 75 73 5f 61 8144f598>] bus_a
> [ 77.708758] Object 800000002e938a60: 64 64 5f 64 72 69 76 65 72 2b
> 30 78 32 30 38 2f dd_driver+0x208/
> [ 77.708768] Object 800000002e938a70: 30 78 32 38 30 0a 20 20 20 20
> 5b 3c 66 66 66 66 0x280. [<ffff
> [ 77.708778] Object 800000002e938a80: 66 66 66 66 38 31 34 35 31 30
> 30 30 3e 5d 20 64 ffff81451000>] d
> [ 77.708788] Object 800000002e938a90: 72 69 76 65 72 5f 72 65 67 69
> 73 74 65 72 2b 30 river_register+0
> [ 77.708798] Object 800000002e938aa0: 78 39 30 2f 30 78 31 33 38 0a
> 20 20 20 20 5b 3c x90/0x138. [<
> [ 77.708808] Object 800000002e938ab0: 66 66 66 66 66 66 66 66 38 31
> 31 30 30 35 35 30 ffffffff81100550
> [ 77.708818] Object 800000002e938ac0: 3e 5d 20 64 6f 5f 6f 6e 65 5f
> 69 6e 69 74 63 61 >] do_one_initca
> [ 77.708828] Object 800000002e938ad0: 6c 6c 2b 30 78 61 30 2f 30 78
> 31 63 30 0a 20 20 ll+0xa0/0x1c0.
> [ 77.708838] Object 800000002e938ae0: 20 20 5b 3c 66 66 66 66 66 66
> 66 66 38 31 38 33 [<ffffffff8183
> [ 77.708848] Object 800000002e938af0: 63 65 31 34 3e 5d 20 6b 65 72
> 6e 65 6c 5f 69 6e ce14>] kernel_in
> [ 77.708858] Object 800000002e938b00: 69 74 5f 66 72 65 65 61 62 6c
> 65 2b 30 78 31 38 it_freeable+0x18
> [ 77.708868] Object 800000002e938b10: 38 2f 30 78 32 34 34 0a 20 20
> 20 20 5b 3c 66 66 8/0x244. [<ff
> [ 77.708878] Object 800000002e938b20: 66 66 66 66 66 66 38 31 36 31
> 61 30 31 38 3e 5d ffffff8161a018>]
> [ 77.708888] Object 800000002e938b30: 20 6b 65 72 6e 65 6c 5f 69 6e
> 69 74 2b 30 78 32 kernel_init+0x2
> [ 77.708897] Object 800000002e938b40: 30 2f 30 78 31 31 38 0a 20 20
> 20 20 5b 3c 66 66 0/0x118. [<ff
> [ 77.708907] Object 800000002e938b50: 66 66 66 66 66 66 38 31 31 32
> 30 61 36 63 3e 5d ffffff81120a6c>]
> [ 77.708917] Object 800000002e938b60: 20 72 65 74 5f 66 72 6f 6d 5f
> 6b 65 72 6e 65 6c ret_from_kernel
> [ 77.708927] Object 800000002e938b70: 5f 74 68 72 65 61 64 2b 30 78
> 31 34 2f 30 78 31 _thread+0x14/0x1
> [ 77.708937] Object 800000002e938b80: 63 0a 75 6e 72 65 66 65 72 65
> 6e 63 65 64 20 6f c.unreferenced o
> [ 77.708947] Object 800000002e938b90: 62 6a 65 63 74 20 30 78 38 30
> 30 30 30 30 30 30 bject 0x80000000
> [ 77.708957] Object 800000002e938ba0: 32 66 33 37 34 61 34 30 20 28
> 73 69 7a 65 20 32 2f374a40 (size 2
> [ 77.708967] Object 800000002e938bb0: 30 34 38 29 3a 0a 20 20 63 6f
> 6d 6d 20 22 73 77 048):. comm "sw
> [ 77.708977] Object 800000002e938bc0: 61 70 70 65 72 2f 30 22 2c 20
> 70 69 64 20 31 2c apper/0", pid 1,
> [ 77.708987] Object 800000002e938bd0: 20 6a 69 66 66 69 65 73 20 34
> 32 39 34 39 33 38 jiffies 4294938
> [ 77.708997] Object 800000002e938be0: 30 35 31 20 28 61 67 65 20 34
> 31 2e 35 37 30 73 051 (age 41.570s
> [ 77.709007] Object 800000002e938bf0: 29 0a 20 20 68 65 78 20 64 75
> 6d 70 20 28 66 69 ). hex dump (fi
> [ 77.709017] Object 800000002e938c00: 72 73 74 20 33 32 20 62 79 74
> 65 73 29 3a 0a 20 rst 32 bytes):.
> [ 77.709027] Object 800000002e938c10: 20 20 20 36 62 20 36 62 20 36
> 62 20 36 62 20 36 6b 6b 6b 6b 6
> [ 77.709037] Object 800000002e938c20: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.709047] Object 800000002e938c30: 20 36 62 20 36 62 20 36 62 20
> 36 62 20 36 62 20 6b 6b 6b 6b 6b
> [ 77.709056] Object 800000002e938c40: 36 62 20 20 6b 6b 6b 6b 6b 6b
> 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkk
> [ 77.709066] Object 800000002e938c50: 6b 6b 6b 6b 0a 20 20 20 20 36
> 62 20 36 62 20 36 kkkk. 6b 6b 6
> [ 77.709076] Object 800000002e938c60: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 36 62 b 6b 6b 6b 6b 6b
> [ 77.709086] Object 800000002e938c70: 20 36 62 20 36 62 20 36 62 20
> 36 62 20 36 62 20 6b 6b 6b 6b 6b
> [ 77.709096] Object 800000002e938c80: 36 62 20 36 62 20 36 62 20 20
> 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkk
> [ 77.709106] Object 800000002e938c90: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
> 0a 20 20 62 61 63 kkkkkkkkkk. bac
> [ 77.709116] Object 800000002e938ca0: 6b 74 72 61 63 65 3a 0a 20 20
> 20 20 5b 3c 66 66 ktrace:. [<ff
> [ 77.709126] Object 800000002e938cb0: 66 66 66 66 66 66 38 31 32 36
> 63 39 62 38 3e 5d ffffff8126c9b8>]
> [ 77.709136] Object 800000002e938cc0: 20 5f 5f 6b 6d 61 6c 6c 6f 63
> 2b 30 78 31 62 38 __kmalloc+0x1b8
> [ 77.709146] Object 800000002e938cd0: 2f 30 78 33 66 30 0a 20 20 20
> 20 5b 3c 66 66 66 /0x3f0. [<fff
> [ 77.709156] Object 800000002e938ce0: 66 66 66 66 66 38 31 34 65 31
> 31 62 38 3e 5d 20 fffff814e11b8>]
> [ 77.709166] Object 800000002e938cf0: 63 76 6d 5f 6f 63 74 5f 6d 65
> 6d 5f 66 69 6c 6c cvm_oct_mem_fill
> [ 77.709176] Object 800000002e938d00: 5f 66 70 61 2b 30 78 37 38 2f
> 30 78 31 64 38 0a _fpa+0x78/0x1d8.
> [ 77.709186] Object 800000002e938d10: 20 20 20 20 5b 3c 66 66 66 66
> 66 66 66 66 38 31 [<ffffffff81
> [ 77.709196] Object 800000002e938d20: 34 64 66 64 33 38 3e 5d 20 63
> 76 6d 5f 6f 63 74 4dfd38>] cvm_oct
> [ 77.709206] Object 800000002e938d30: 5f 70 72 6f 62 65 2b 30 78 66
> 38 2f 30 78 37 62 _probe+0xf8/0x7b
> [ 77.709216] Object 800000002e938d40: 30 0a 20 20 20 20 5b 3c 66 66
> 66 66 66 66 66 66 0. [<ffffffff
> [ 77.709225] Object 800000002e938d50: 38 31 34 35 32 37 61 38 3e 5d
> 20 70 6c 61 74 66 814527a8>] platf
> [ 77.709235] Object 800000002e938d60: 6f 72 6d 5f 64 72 76 5f 70 72
> 6f 62 65 2b 30 78 orm_drv_probe+0x
> [ 77.709245] Object 800000002e938d70: 34 38 2f 30 78 64 38 0a 20 20
> 20 20 5b 3c 66 66 48/0xd8. [<ff
> [ 77.709255] Object 800000002e938d80: 66 66 66 66 66 66 38 31 34 35
> 30 31 34 63 3e 5d ffffff8145014c>]
> [ 77.709265] Object 800000002e938d90: 20 64 72 69 76 65 72 5f 70 72
> 6f 62 65 5f 64 65 driver_probe_de
> [ 77.709275] Object 800000002e938da0: 76 69 63 65 2b 30 78 32 39 63
> 2f 30 78 33 35 30 vice+0x29c/0x350
> [ 77.709285] Object 800000002e938db0: 0a 20 20 20 20 5b 3c 66 66 66
> 66 66 66 66 66 38 . [<ffffffff8
> [ 77.709295] Object 800000002e938dc0: 31 34 35 30 32 64 30 3e 5d 20
> 5f 5f 64 72 69 76 14502d0>] __driv
> [ 77.709305] Object 800000002e938dd0: 65 72 5f 61 74 74 61 63 68 2b
> 30 78 64 30 2f 30 er_attach+0xd0/0
> [ 77.709315] Object 800000002e938de0: 78 64 38 0a 20 20 20 20 5b 3c
> 66 66 66 66 66 66 xd8. [<ffffff
> [ 77.709325] Object 800000002e938df0: 66 66 38 31 34 34 64 63 32 34
> 3e 5d 20 62 75 73 ff8144dc24>] bus
> [ 77.709335] Object 800000002e938e00: 5f 66 6f 72 5f 65 61 63 68 5f
> 64 65 76 2b 30 78 _for_each_dev+0x
> [ 77.709345] Object 800000002e938e10: 37 34 2f 30 78 63 30 0a 20 20
> 20 20 5b 3c 66 66 74/0xc0. [<ff
> [ 77.709355] Object 800000002e938e20: 66 66 66 66 66 66 38 31 34 34
> 66 35 39 38 3e 5d ffffff8144f598>]
> [ 77.709365] Object 800000002e938e30: 20 62 75 73 5f 61 64 64 5f 64
> 72 69 76 65 72 2b bus_add_driver+
> [ 77.709375] Object 800000002e938e40: 30 78 32 30 38 2f 30 78 32 38
> 30 0a 20 20 20 20 0x208/0x280.
> [ 77.709385] Object 800000002e938e50: 5b 3c 66 66 66 66 66 66 66 66
> 38 31 34 35 31 30 [<ffffffff814510
> [ 77.709394] Object 800000002e938e60: 30 30 3e 5d 20 64 72 69 76 65
> 72 5f 72 65 67 69 00>] driver_regi
> [ 77.709404] Object 800000002e938e70: 73 74 65 72 2b 30 78 39 30 2f
> 30 78 31 33 38 0a ster+0x90/0x138.
> [ 77.709414] Object 800000002e938e80: 20 20 20 20 5b 3c 66 66 66 66
> 66 66 66 66 38 31 [<ffffffff81
> [ 77.709424] Object 800000002e938e90: 31 30 30 35 35 30 3e 5d 20 64
> 6f 5f 6f 6e 65 5f 100550>] do_one_
> [ 77.709434] Object 800000002e938ea0: 69 6e 69 74 63 61 6c 6c 2b 30
> 78 61 30 2f 30 78 initcall+0xa0/0x
> [ 77.709444] Object 800000002e938eb0: 31 63 30 0a 20 20 20 20 5b 3c
> 66 66 66 66 66 66 1c0. [<ffffff
> [ 77.709454] Object 800000002e938ec0: 66 66 38 31 38 33 63 65 31 34
> 3e 5d 20 6b 65 72 ff8183ce14>] ker
> [ 77.709464] Object 800000002e938ed0: 6e 65 6c 5f 69 6e 69 74 5f 66
> 72 65 65 61 62 6c nel_init_freeabl
> [ 77.709474] Object 800000002e938ee0: 65 2b 30 78 31 38 38 2f 30 78
> 32 34 34 0a 20 20 e+0x188/0x244.
> [ 77.709484] Object 800000002e938ef0: 20 20 5b 3c 66 66 66 66 66 66
> 66 66 38 31 36 31 [<ffffffff8161
> [ 77.709494] Object 800000002e938f00: 61 30 31 38 3e 5d 20 6b 65 72
> 6e 65 6c 5f 69 6e a018>] kernel_in
> [ 77.709504] Object 800000002e938f10: 69 74 2b 30 78 32 30 2f 30 78
> 31 31 38 0a 20 20 it+0x20/0x118.
> [ 77.709514] Object 800000002e938f20: 20 20 5b 3c 66 66 66 66 66 66
> 66 66 38 31 31 32 [<ffffffff8112
> [ 77.709524] Object 800000002e938f30: 30 61 36 63 3e 5d 20 72 65 74
> 5f 66 72 6f 6d 5f 0a6c>] ret_from_
> [ 77.709534] Object 800000002e938f40: 6b 65 72 6e 65 6c 5f 74 68 72
> 65 61 64 2b 30 78 kernel_thread+0x
> [ 77.709544] Object 800000002e938f50: 31 34 2f 30 78 31 63 0a 75 6e
> 72 65 66 65 72 65 14/0x1c.unrefere
> [ 77.709553] Object 800000002e938f60: 6e 63 65 64 20 6f 62 6a 65 63
> 74 20 30 78 38 30 nced object 0x80
> [ 77.709563] Object 800000002e938f70: 30 30 30 30 30 30 32 66 33 37
> 32 65 36 38 20 28 0000002f372e68 (
> [ 77.709573] Object 800000002e938f80: 73 69 7a 65 20 32 30 34 38 29
> 3a 0a 20 20 63 6f size 2048):. co
> [ 77.709583] Object 800000002e938f90: 6d 6d 20 22 73 77 61 70 70 65
> 72 2f 30 22 2c 20 mm "swapper/0",
> [ 77.709593] Object 800000002e938fa0: 70 69 64 20 31 2c 20 6a 69 66
> 66 69 65 73 20 34 pid 1, jiffies 4
> [ 77.709603] Object 800000002e938fb0: 32 39 34 39 33 38 30 35 31 20
> 28 61 67 65 20 34 294938051 (age 4
> [ 77.709613] Object 800000002e938fc0: 31 2e 35 37 30 73 29 0a 20 20
> 68 65 78 20 64 75 1.570s). hex du
> [ 77.709623] Object 800000002e938fd0: 6d 70 20 28 66 69 72 73 74 20
> 33 32 20 62 79 74 mp (first 32 byt
> [ 77.709633] Object 800000002e938fe0: 65 73 29 3a 0a 20 20 20 20 36
> 62 20 36 62 20 36 es):. 6b 6b 6
> [ 77.709643] Object 800000002e938ff0: 62 20 36 62 20 36 62 20 36 62
> 20 36 62 20 00 20 b 6b 6b 6b 6b .
> [ 77.709653] Redzone 800000002e939000: 00 cc cc cc cc cc cc
> cc ........
> [ 77.709663] Padding 800000002e939140: 5a 5a 5a 5a 5a 5a 5a
> 5a ZZZZZZZZ
> [ 77.709675] CPU: 2 PID: 587 Comm: wc Tainted:
> G B 4.3.0-rc7-octeon-distro.git-v1.14-dirty #1
> [ 77.709682] Stack : 0000000000000000 ffffffff817c0000
> ffffffff81f50000 ffffffff817c6c18
> [ 77.709682] 0000000000000002 0000000000000004
> ffffffff817c0000 0000000000000000
> [ 77.709682] ffffffff81f50000 0000000000000002
> 0000000000000004 0000000000000002
> [ 77.709682] 0000000000000004 ffffffff81199a4c
> 0000000000000000 0000000000000000
> [ 77.709682] ffffffff81f50000 0000000000000009
> ffffffff816f2990 ffffffff81199eb8
> [ 77.709682] 800000002fad0e48 ffffffff81f47bf8
> 000000000000024b 0000000000000002
> [ 77.709682] 8000000003ec4440 ffffffff816ffc20
> 000000000003c400 ffffffff8121641c
> [ 77.709682] 800000002f8bbb98 800000002f8bba80
> 8000000003ec4440 ffffffff813c41e0
> [ 77.709682] ffffffff816eb028 ffffffff8119ab9c
> 000000000000005a ffffffff816eb028
> [ 77.709682] 0000000000000002 ffffffff81127020
> 0000000000000000 0000000000000000
> [ 77.709682] ...
> [ 77.709819] Call Trace:
> [ 77.709829] [<ffffffff81127020>] show_stack+0x98/0xb8
> [ 77.709843] [<ffffffff813c41e0>] dump_stack+0x80/0xd0
> [ 77.709856] [<ffffffff81268450>]
> check_bytes_and_report+0x118/0x150
> [ 77.709867] [<ffffffff812686a0>] check_object+0x218/0x2e8
> [ 77.709879] [<ffffffff81269e04>] free_debug_processing+0x1e4/0x3f8
> [ 77.709890] [<ffffffff8126cf30>] __slab_free+0x340/0x4f0
> [ 77.709901] [<ffffffff812a68b4>] seq_release+0x24/0x40
> [ 77.709912] [<ffffffff8127f02c>] __fput+0xa4/0x218
> [ 77.709924] [<ffffffff81167678>] task_work_run+0xb0/0x108
> [ 77.709935] [<ffffffff81120c88>] work_notifysig+0x10/0x18
> [ 77.709943]
> [ 77.709951] FIX kmalloc-4096: Restoring 0x800000002e939000-
> 0x800000002e939000=0xcc
> [ 77.709951]
--
Andy Shevchenko <andriy.shevchenko@...ux.intel.com>
Intel Finland Oy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists