| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <Y2ElURkvmGD5csMc@feng-clx>
Date: Tue, 1 Nov 2022 21:55:29 +0800
From: Feng Tang <feng.tang@...el.com>
To: Hyeonggon Yoo <42.hyeyoo@...il.com>
CC: John Thomson <lists@...nthomson.fastmail.com.au>,
Vlastimil Babka <vbabka@...e.cz>,
Andrew Morton <akpm@...ux-foundation.org>,
"Christoph Lameter" <cl@...ux.com>,
Pekka Enberg <penberg@...nel.org>,
David Rientjes <rientjes@...gle.com>,
Joonsoo Kim <iamjoonsoo.kim@....com>,
Roman Gushchin <roman.gushchin@...ux.dev>,
Dmitry Vyukov <dvyukov@...gle.com>,
"Jonathan Corbet" <corbet@....net>,
Andrey Konovalov <andreyknvl@...il.com>,
"Hansen, Dave" <dave.hansen@...el.com>,
"linux-mm@...ck.org" <linux-mm@...ck.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"kasan-dev@...glegroups.com" <kasan-dev@...glegroups.com>,
Robin Murphy <robin.murphy@....com>,
John Garry <john.garry@...wei.com>,
Kefeng Wang <wangkefeng.wang@...wei.com>,
Thomas Bogendoerfer <tsbogend@...ha.franken.de>,
John Crispin <john@...ozen.org>,
"Matthias Brugger" <matthias.bgg@...il.com>,
"linux-mips@...r.kernel.org" <linux-mips@...r.kernel.org>
Subject: Re: [PATCH v6 1/4] mm/slub: enable debugging memory wasting of
kmalloc
On Tue, Nov 01, 2022 at 06:42:23PM +0800, Hyeonggon Yoo wrote:
> On Tue, Nov 01, 2022 at 10:33:32AM +0000, John Thomson wrote:
> > On Tue, 1 Nov 2022, at 09:31, Hyeonggon Yoo wrote:
> > > On Tue, Nov 01, 2022 at 09:20:21AM +0000, John Thomson wrote:
> > >> On Tue, 1 Nov 2022, at 07:57, Feng Tang wrote:
> > >> > Hi Thomson,
> > >> >
> > >> > Thanks for testing!
> > >> >
> > >> > + mips maintainer and mail list. The original report is here
> > >> >
> > >> > https://lore.kernel.org/lkml/becf2ac3-2a90-4f3a-96d9-a70f67c66e4a@app.fastmail.com/
> > >>
> > >> I am guessing my issue comes from __kmem_cache_alloc_lru accessing s->object_size when (kmem_cache) s is NULL?
> > >> If that is the case, this change is not to blame, it only exposes the issue?
> > >>
> > >> I get the following dmesg (note very early NULL kmem_cache) with the below change atop v6.1-rc3:
> > >>
> > >> transfer started ......................................... transfer ok, time=2.02s
> > >> setting up elf image... OK
> > >> jumping to kernel code
> > >> zimage at: 80B842A0 810B4EFC
> > >>
> > >> Uncompressing Linux at load address 80001000
> > >>
> > >> Copy device tree to address 80B80EE0
> > >>
> > >> Now, booting the kernel...
> > >>
> > >> [ 0.000000] Linux version 6.1.0-rc3+ (john@...n) (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0, GNU ld (GNU Binutils) 2.39) #61 SMP Tue Nov 1 18:04:13 AEST 2022
> > >> [ 0.000000] slub: kmem_cache_alloc called with kmem_cache: 0x0
> > >> [ 0.000000] slub: __kmem_cache_alloc_lru called with kmem_cache: 0x0
> > >> [ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
> > >> [ 0.000000] printk: bootconsole [early0] enabled
> > >> [ 0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
> > >> [ 0.000000] MIPS: machine is MikroTik RouterBOARD 760iGS
> > >>
> > >> normal boot
> > >>
> > >>
> > >> diff --git a/mm/slub.c b/mm/slub.c
> > >> index 157527d7101b..10fcdf2520d2 100644
> > >> --- a/mm/slub.c
> > >> +++ b/mm/slub.c
> > >> @@ -3410,7 +3410,13 @@ static __always_inline
> > >> void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
> > >> gfp_t gfpflags)
> > >> {
> > >> - void *ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
> > >> + void *ret;
> > >> + if (IS_ERR_OR_NULL(s)) {
> > >> + pr_warn("slub: __kmem_cache_alloc_lru called with kmem_cache: %pSR\n", s);
> > >> + ret = slab_alloc(s, lru, gfpflags, _RET_IP_, 0);
> > >> + } else {
> > >> + ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
> > >> + }
> > >>
> > >> trace_kmem_cache_alloc(_RET_IP_, ret, s, gfpflags, NUMA_NO_NODE);
> > >>
> > >> @@ -3419,6 +3425,8 @@ void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
> > >>
> > >> void *kmem_cache_alloc(struct kmem_cache *s, gfp_t gfpflags)
> > >> {
> > >> + if (IS_ERR_OR_NULL(s))
> > >> + pr_warn("slub: kmem_cache_alloc called with kmem_cache: %pSR\n", s);
> > >> return __kmem_cache_alloc_lru(s, NULL, gfpflags);
> > >> }
> > >> EXPORT_SYMBOL(kmem_cache_alloc);
> > >> @@ -3426,6 +3434,8 @@ EXPORT_SYMBOL(kmem_cache_alloc);
> > >> void *kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
> > >> gfp_t gfpflags)
> > >> {
> > >> + if (IS_ERR_OR_NULL(s))
> > >> + pr_warn("slub: __kmem_cache_alloc_lru called with kmem_cache: %pSR\n", s);
> > >> return __kmem_cache_alloc_lru(s, lru, gfpflags);
> > >> }
> > >> EXPORT_SYMBOL(kmem_cache_alloc_lru);
> > >>
> > >>
> > >> Any hints on where kmem_cache_alloc would be being called from this early?
> > >> I will start looking from /init/main.c around pr_notice("%s", linux_banner);
> > >
> > > Great. Would you try calling dump_stack(); when we observed s == NULL?
> > > That would give more information about who passed s == NULL to these
> > > functions.
> > >
> >
> > With the dump_stack() in place:
> >
> > Now, booting the kernel...
> >
> > [ 0.000000] Linux version 6.1.0-rc3+ (john@...n) (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0, GNU ld (GNU Binutils) 2.39) #62 SMP Tue Nov 1 19:49:52 AEST 2022
> > [ 0.000000] slub: __kmem_cache_alloc_lru called with kmem_cache ptr: 0x0
> > [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.1.0-rc3+ #62
> > [ 0.000000] Stack : 810fff78 80084d98 80889d00 00000004 00000000 00000000 80889d5c 80c90000
> > [ 0.000000] 80920000 807bd380 8089d368 80923bd3 00000000 00000001 80889d08 00000000
> > [ 0.000000] 00000000 00000000 807bd380 8084bd51 00000002 00000002 00000001 6d6f4320
> > [ 0.000000] 00000000 80c97ce9 80c97d14 fffffffc 807bd380 00000000 00000003 00000dc0
> > [ 0.000000] 00000000 a0000000 80910000 8110a0b4 00000000 00000020 80010000 80010000
> > [ 0.000000] ...
> > [ 0.000000] Call Trace:
> > [ 0.000000] [<80008260>] show_stack+0x28/0xf0
> > [ 0.000000] [<8070cdc0>] dump_stack_lvl+0x60/0x80
> > [ 0.000000] [<801c1428>] kmem_cache_alloc+0x5c0/0x740
> > [ 0.000000] [<8092856c>] prom_soc_init+0x1fc/0x2b4
> > [ 0.000000] [<80928060>] prom_init+0x44/0xf0
> > [ 0.000000] [<80929214>] setup_arch+0x4c/0x6a8
> > [ 0.000000] [<809257e0>] start_kernel+0x88/0x7c0
> > [ 0.000000]
> > [ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
>
> setup_arch() is too early to use slab allocators.
> I think slab received NULL pointer because kmalloc is not initialized.
>
> It seems arch/mips/ralink/mt7621.c is using slab too early.
Cool! it is finally root caused :) Thanks!
The following patch should solve it and give it a warning message, though
I'm not sure if there is other holes.
Thanks,
Feng
---
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 33b1886b06eb..429c21b7ecbc 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1043,7 +1043,14 @@ size_t __ksize(const void *object)
#ifdef CONFIG_TRACING
void *kmalloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size)
{
- void *ret = __kmem_cache_alloc_node(s, gfpflags, NUMA_NO_NODE,
+ void *ret;
+
+ if (unlikely(ZERO_OR_NULL_PTR(s))) {
+ WARN_ON_ONCE(1);
+ return s;
+ }
+
+ ret = __kmem_cache_alloc_node(s, gfpflags, NUMA_NO_NODE,
size, _RET_IP_);
trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags, NUMA_NO_NODE);
diff --git a/mm/slub.c b/mm/slub.c
index 157527d7101b..85d24bb6eda7 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3410,8 +3410,14 @@ static __always_inline
void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
gfp_t gfpflags)
{
- void *ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
+ void *ret;
+ if (unlikely(ZERO_OR_NULL_PTR(s))) {
+ WARN_ON_ONCE(1);
+ return s;
+ }
+
+ ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
trace_kmem_cache_alloc(_RET_IP_, ret, s, gfpflags, NUMA_NO_NODE);
return ret;
Powered by blists - more mailing lists