| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20190619030940.GA17244@castle.DHCP.thefacebook.com>
Date: Wed, 19 Jun 2019 03:09:53 +0000
From: Roman Gushchin <guro@...com>
To: Qian Cai <cai@....pw>
CC: Vladimir Davydov <vdavydov.dev@...il.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Linux MM <linux-mm@...ck.org>,
"Linux Kernel Mailing List" <linux-kernel@...r.kernel.org>,
Johannes Weiner <hannes@...xchg.org>
Subject: Re: "mm: reparent slab memory on cgroup removal" series triggers
SLUB_DEBUG errors
On Tue, Jun 18, 2019 at 05:43:04PM -0400, Qian Cai wrote:
> Booting linux-next on both arm64 and powerpc triggers SLUB_DEBUG errors below. Reverted the whole series “mm: reparent slab memory on cgroup removal” [1] fixed the issue.
Hi Qian!
Thank you for the report!
Didn't you try to reproduce it on x86? All the code changed in this series
isn't arch-specific, so if it can be seen only on ppc and arm64, that's
interesting.
I'm currently on PTO and have a very limited internet connection,
so I won't be able to reproduce the issue up to Sunday, when I'll be back.
If you can try reverting only the last patch from the series,
I will appreciate it.
Thanks!
>
> [1] https://lore.kernel.org/lkml/20190611231813.3148843-1-guro@fb.com/
>
> [ 151.773224][ T1650] BUG kmem_cache (Tainted: G B W ): Poison overwritten
> [ 151.780969][ T1650] -----------------------------------------------------------------------------
> [ 151.780969][ T1650]
> [ 151.792016][ T1650] INFO: 0x000000001fd6fdef-0x0000000007f6bb36. First byte 0x0 instead of 0x6b
> [ 151.800726][ T1650] INFO: Allocated in create_cache+0x6c/0x1bc age=24301 cpu=97 pid=1444
> [ 151.808821][ T1650] kmem_cache_alloc+0x514/0x568
> [ 151.813527][ T1650] create_cache+0x6c/0x1bc
> [ 151.817800][ T1650] memcg_create_kmem_cache+0xfc/0x11c
> [ 151.823028][ T1650] memcg_kmem_cache_create_func+0x40/0x170
> [ 151.828691][ T1650] process_one_work+0x4e0/0xa54
> [ 151.833398][ T1650] worker_thread+0x498/0x650
> [ 151.837843][ T1650] kthread+0x1b8/0x1d4
> [ 151.841770][ T1650] ret_from_fork+0x10/0x18
> [ 151.846046][ T1650] INFO: Freed in slab_kmem_cache_release+0x3c/0x48 age=23341 cpu=28 pid=1480
> [ 151.854659][ T1650] slab_kmem_cache_release+0x3c/0x48
> [ 151.859799][ T1650] kmem_cache_release+0x1c/0x28
> [ 151.864507][ T1650] kobject_cleanup+0x134/0x288
> [ 151.869127][ T1650] kobject_put+0x5c/0x68
> [ 151.873226][ T1650] sysfs_slab_release+0x2c/0x38
> [ 151.877931][ T1650] shutdown_cache+0x198/0x23c
> [ 151.882464][ T1650] kmemcg_cache_shutdown_fn+0x1c/0x34
> [ 151.887691][ T1650] kmemcg_workfn+0x44/0x68
> [ 151.891963][ T1650] process_one_work+0x4e0/0xa54
> [ 151.896668][ T1650] worker_thread+0x498/0x650
> [ 151.901113][ T1650] kthread+0x1b8/0x1d4
> [ 151.905037][ T1650] ret_from_fork+0x10/0x18
> [ 151.909324][ T1650] INFO: Slab 0x00000000406d65a6 objects=64 used=64 fp=0x000000004d988e71 flags=0x7ffffffc000200
> [ 151.919596][ T1650] INFO: Object 0x0000000040f4b79e @offset=15420325124116637824 fp=0x00000000e038adbf
> [ 151.919596][ T1650]
> [ 151.931079][ T1650] Redzone 00000000fc4c04f0: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 151.941168][ T1650] Redzone 000000009a25c019: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 151.951256][ T1650] Redzone 000000000b05c7cc: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 151.961345][ T1650] Redzone 00000000a08ae38b: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 151.971433][ T1650] Redzone 00000000e0eccd41: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 151.981520][ T1650] Redzone 0000000016ee2661: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 151.991608][ T1650] Redzone 000000009364e729: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 152.001695][ T1650] Redzone 00000000f2202456: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb ................
> [ 152.011784][ T1650] Object 0000000040f4b79e: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.021783][ T1650] Object 000000002df21fec: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.031779][ T1650] Object 0000000041cf0887: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.041775][ T1650] Object 00000000bfb91e8f: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.051770][ T1650] Object 00000000da315b1c: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.061765][ T1650] Object 00000000b362de78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.071761][ T1650] Object 00000000ad4f72bf: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.081756][ T1650] Object 00000000aa32d346: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.091751][ T1650] Object 00000000ad1cf22c: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.101746][ T1650] Object 000000001cee47e4: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.111741][ T1650] Object 00000000418720ed: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.121736][ T1650] Object 00000000dee1c3f2: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.131731][ T1650] Object 00000000a23397c1: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.141727][ T1650] Object 000000002ed01641: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.151721][ T1650] Object 00000000915ec720: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.161716][ T1650] Object 00000000915988c1: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.171711][ T1650] Object 000000004a0cc60f: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.181707][ T1650] Object 0000000054a294c9: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.191701][ T1650] Object 0000000054f61682: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.201697][ T1650] Object 0000000018d04328: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.211692][ T1650] Object 00000000703cf2c7: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.221687][ T1650] Object 000000004d3ac5d5: 6b 6b 6b 6b 6b 6b 6b 6b 00 00 00 00 00 00 00 00 kkkkkkkk........
> [ 152.231682][ T1650] Object 00000000726ce587: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.241676][ T1650] Object 00000000c709b64e: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.251672][ T1650] Object 0000000044d6a5c6: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.261667][ T1650] Object 000000009c76a6a2: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.271662][ T1650] Object 0000000033d01d12: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.281657][ T1650] Object 00000000c50ff26f: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.291652][ T1650] Object 00000000ebc3aaae: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.301647][ T1650] Object 00000000a2072fe3: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> [ 152.311641][ T1650] Object 000000003d5911a3: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
> [ 152.320942][ T1650] Redzone 000000009a2feac1: bb bb bb bb bb bb bb bb ........
> [ 152.330330][ T1650] Padding 00000000c1b3cb8b: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [ 152.340412][ T1650] Padding 000000003715421a: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [ 152.350493][ T1650] Padding 0000000066b51ba7: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [ 152.360575][ T1650] Padding 00000000ca240306: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> [ 152.370657][ T1650] Padding 0000000014a2af5d: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
> [ 152.380048][ T1650] CPU: 82 PID: 1650 Comm: kworker/82:1 Tainted: G B W 5.2.0-rc5-next-20190617 #18
> [ 152.390216][ T1650] Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.0.9 03/01/2019
> [ 152.400741][ T1650] Workqueue: memcg_kmem_cache memcg_kmem_cache_create_func
> [ 152.407786][ T1650] Call trace:
> [ 152.410926][ T1650] dump_backtrace+0x0/0x268
> [ 152.415280][ T1650] show_stack+0x20/0x2c
> [ 152.419287][ T1650] dump_stack+0xb4/0x108
> [ 152.423384][ T1650] print_trailer+0x274/0x298
> [ 152.427825][ T1650] check_bytes_and_report+0xc4/0x118
> [ 152.432959][ T1650] check_object+0x2fc/0x36c
> [ 152.437312][ T1650] alloc_debug_processing+0x154/0x240
> [ 152.442532][ T1650] ___slab_alloc+0x710/0xa68
> [ 152.446972][ T1650] kmem_cache_alloc+0x514/0x568
> [ 152.451672][ T1650] create_cache+0x6c/0x1bc
> [ 152.455938][ T1650] memcg_create_kmem_cache+0xfc/0x11c
> [ 152.461158][ T1650] memcg_kmem_cache_create_func+0x40/0x170
> [ 152.466814][ T1650] process_one_work+0x4e0/0xa54
> [ 152.471515][ T1650] worker_thread+0x498/0x650
> [ 152.475953][ T1650] kthread+0x1b8/0x1d4
> [ 152.479872][ T1650] ret_from_fork+0x10/0x18
> [ 152.484139][ T1650] FIX kmem_cache: Restoring 0x000000001fd6fdef-0x0000000007f6bb36=0x6b
> [ 152.484139][ T1650]
> [ 152.494395][ T1650] FIX kmem_cache: Marking all objects used
Powered by blists - more mailing lists