linux-kernel - Re: [PATCH v6 1/4] mm/slub: enable debugging memory wasting of kmalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <f88a5d34-de05-25d7-832d-36b3a3eddd72@suse.cz>
Date:   Thu, 3 Nov 2022 09:33:28 +0100
From:   Vlastimil Babka <vbabka@...e.cz>
To:     Feng Tang <feng.tang@...el.com>
Cc:     John Thomson <lists@...nthomson.fastmail.com.au>,
        Hyeonggon Yoo <42.hyeyoo@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Christoph Lameter <cl@...ux.com>,
        Pekka Enberg <penberg@...nel.org>,
        David Rientjes <rientjes@...gle.com>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Jonathan Corbet <corbet@....net>,
        Andrey Konovalov <andreyknvl@...il.com>,
        "Hansen, Dave" <dave.hansen@...el.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "kasan-dev@...glegroups.com" <kasan-dev@...glegroups.com>,
        Robin Murphy <robin.murphy@....com>,
        John Garry <john.garry@...wei.com>,
        Kefeng Wang <wangkefeng.wang@...wei.com>,
        Thomas Bogendoerfer <tsbogend@...ha.franken.de>,
        "linux-mips@...r.kernel.org" <linux-mips@...r.kernel.org>
Subject: Re: [PATCH v6 1/4] mm/slub: enable debugging memory wasting of
 kmalloc

On 11/3/22 06:54, Feng Tang wrote:
> On Wed, Nov 02, 2022 at 04:22:37PM +0800, Vlastimil Babka wrote:
>> On 11/1/22 11:33, John Thomson wrote:
> [...]
>> > 
>> > [    0.000000] Linux version 6.1.0-rc3+ (john@...n) (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0, GNU ld (GNU Binutils) 2.39) #62 SMP Tue Nov  1 19:49:52 AEST 2022
>> > [    0.000000] slub: __kmem_cache_alloc_lru called with kmem_cache ptr: 0x0
>> > [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.1.0-rc3+ #62
>> > [    0.000000] Stack : 810fff78 80084d98 80889d00 00000004 00000000 00000000 80889d5c 80c90000
>> > [    0.000000]         80920000 807bd380 8089d368 80923bd3 00000000 00000001 80889d08 00000000
>> > [    0.000000]         00000000 00000000 807bd380 8084bd51 00000002 00000002 00000001 6d6f4320
>> > [    0.000000]         00000000 80c97ce9 80c97d14 fffffffc 807bd380 00000000 00000003 00000dc0
>> > [    0.000000]         00000000 a0000000 80910000 8110a0b4 00000000 00000020 80010000 80010000
>> > [    0.000000]         ...
>> > [    0.000000] Call Trace:
>> > [    0.000000] [<80008260>] show_stack+0x28/0xf0
>> > [    0.000000] [<8070cdc0>] dump_stack_lvl+0x60/0x80
>> > [    0.000000] [<801c1428>] kmem_cache_alloc+0x5c0/0x740
>> > [    0.000000] [<8092856c>] prom_soc_init+0x1fc/0x2b4
>> > [    0.000000] [<80928060>] prom_init+0x44/0xf0
>> > [    0.000000] [<80929214>] setup_arch+0x4c/0x6a8
>> > [    0.000000] [<809257e0>] start_kernel+0x88/0x7c0
>> > [    0.000000] 
>> > [    0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
>> 
>> The stack means CONFIG_TRACING=n, is that right?
>  
> Yes, from the kconfig, CONFIG_TRACING is not set.
> 
>> That would mean
>> prom_soc_init()
>>   soc_dev_init()
>>     kzalloc() -> kmalloc()
>>       kmalloc_trace()  // after #else /* CONFIG_TRACING */
>>         kmem_cache_alloc(s, flags);
>> 
>> Looks like this path is a small bug in the wasting detection patch, as we
>> throw away size there.
> 
> Yes, from the code reading and log from John, it is.
> 
> One strange thing is, I reset the code to v6.0, and found that 
> __kmem_cache_alloc_lru() also access the 's->object_size'
> 
> void *__kmem_cache_alloc_lru(struct kmem_cache *s, struct list_lru *lru,
> 			     gfp_t gfpflags)
> {
> 	void *ret = slab_alloc(s, lru, gfpflags, _RET_IP_, s->object_size);
> 	...
> }
> 
> And from John's dump_stack() info, this call is also where the NULL pointer
> happens, which I still can't figue out.
> 
>> AFAICS before this patch, we "survive" "kmem_cache *s" being NULL as
>> slab_pre_alloc_hook() will happen to return NULL and we bail out from
>> slab_alloc_node(). But this is a side-effect, not an intended protection.
>> Also the CONFIG_TRACING variant of kmalloc_trace() would have called
>> trace_kmalloc dereferencing s->size anyway even before this patch.
>> 
>> I don't think we should add WARNS in the slab hot paths just to prevent this
>> rare error of using slab too early. At most VM_WARN... would be acceptable
>> but still not necessary as crashing immediately from a NULL pointer is
>> sufficient.
>> 
>> So IMHO mips should fix their soc init, 
> 
> Yes, for the mips fix, John has proposed to defer the calling of prom_soc_init(),
> which looks reasonable.
> 
>> and we should look into the
>> CONFIG_TRACING=n variant of kmalloc_trace(), to pass orig_size properly.
> 
> You mean check if the pointer is NULL and bail out early. 

No I mean here:

#else /* CONFIG_TRACING */
/* Save a function call when CONFIG_TRACING=n */
static __always_inline __alloc_size(3)                                   
void *kmalloc_trace(struct kmem_cache *s, gfp_t flags, size_t size)
{       
        void *ret = kmem_cache_alloc(s, flags);
                    
        ret = kasan_kmalloc(s, ret, size, flags);
        return ret;
}

we call kmem_cache_alloc() and discard the size parameter, so it will assume
s->object_size (and as the side-effect, crash if s is NULL). We shouldn't
add "s is NULL?" checks, but fix passing the size - probably switch to
__kmem_cache_alloc_node()? and in the following kmalloc_node_trace() analogically.