linux-kernel - Re: [RFC PATCH] mm/slub: enable debugging memory wasting of kmalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <2cf27cd6-5cb9-57e7-dc52-e39f37945343@arm.com>
Date:   Thu, 30 Jun 2022 09:52:12 +0100
From:   Robin Murphy <robin.murphy@....com>
To:     Andrew Morton <akpm@...ux-foundation.org>,
        Feng Tang <feng.tang@...el.com>
Cc:     Christoph Lameter <cl@...ux.com>,
        Pekka Enberg <penberg@...nel.org>,
        David Rientjes <rientjes@...gle.com>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Hyeonggon Yoo <42.hyeyoo@...il.com>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, dave.hansen@...el.com,
        Joerg Roedel <jroedel@...e.de>
Subject: Re: [RFC PATCH] mm/slub: enable debugging memory wasting of kmalloc

On 2022-06-30 03:47, Andrew Morton wrote:
> On Thu, 30 Jun 2022 10:38:44 +0800 Feng Tang <feng.tang@...el.com> wrote:
> 
>> Hi Andrew,
>>
>> Thanks for the review!
>>
>> On Wed, Jun 29, 2022 at 07:30:06PM -0700, Andrew Morton wrote:
>>> On Thu, 30 Jun 2022 09:47:15 +0800 Feng Tang <feng.tang@...el.com> wrote:
>>>
>>>> kmalloc's API family is critical for mm, with one shortcoming that
>>>> its object size is fixed to be power of 2. When user requests memory
>>>> for '2^n + 1' bytes, actually 2^(n+1) bytes will be allocated, so
>>>> in worst case, there is around 50% memory space waste.
>>>>
>>>> We've met a kernel boot OOM panic, and from the dumped slab info:
>>>>
>>>>      [   26.062145] kmalloc-2k            814056KB     814056KB
>>>>
>>>> >From debug we found there are huge number of 'struct iova_magazine',
>>>> whose size is 1032 bytes (1024 + 8), so each allocation will waste
>>>> 1016 bytes. Though the issue is solved by giving the right(bigger)
>>>> size of RAM, it is still better to optimize the size (either use
>>>> a kmalloc friendly size or create a dedicated slab for it).
>>>
>>> Well that's nice, and additional visibility is presumably a good thing.
>>>
>>> But what the heck is going on with iova_magazine?  Is anyone looking at
>>> moderating its impact?
>>
>> Yes, I have a very simple patch at hand
>>
>> --- a/drivers/iommu/iova.c
>> +++ b/drivers/iommu/iova.c
>> @@ -614,7 +614,7 @@ EXPORT_SYMBOL_GPL(reserve_iova);
>>    * dynamic size tuning described in the paper.
>>    */
>>   
>> -#define IOVA_MAG_SIZE 128
>> +#define IOVA_MAG_SIZE 127
> 
> Well OK.  Would benefit from a comment explaining the reasoning.
> 
> But we still have eleventy squillion of these things in flight.  Why?

They're storage for a per-CPU caching scheme - for n CPUs, there should 
currently be (2n + 32) * 6 in flight, since there's one set for each of 
6 sizes. The 32 really should be n or 2n as well since it's needlessly 
large for small systems and a bottleneck for large ones, but it needs 
some unpicking to allow for dynamic allocations.

Thanks,
Robin.