linux-kernel - Re: [RFC 00/15] mm: Implement Slab Movable Objects (SMO)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190312000928.GA25059@tower.DHCP.thefacebook.com>
Date:   Tue, 12 Mar 2019 00:09:31 +0000
From:   Roman Gushchin <guro@...com>
To:     "Tobin C. Harding" <tobin@...nel.org>
CC:     Andrew Morton <akpm@...ux-foundation.org>,
        Christopher Lameter <cl@...ux.com>,
        Pekka Enberg <penberg@...helsinki.fi>,
        Matthew Wilcox <willy@...radead.org>,
        Tycho Andersen <tycho@...ho.ws>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [RFC 00/15] mm: Implement Slab Movable Objects (SMO)

On Fri, Mar 08, 2019 at 03:14:11PM +1100, Tobin C. Harding wrote:
> Hi,
> 
> Here is a patch set implementing movable objects within the SLUB
> allocator.  This is work based on Christopher's patch set:
> 
>  https://lore.kernel.org/patchwork/project/lkml/list/?series=377335
> 
> The original code logic is from that set and implemented by Christopher.
> Clean up, refactoring, documentation, and additional features by myself.
> Blame for any bugs remaining falls solely with myself.  Patches using
> Christopher's code use the Co-developed-by tag.
> 
> After movable objects are implemented a number of useful features become
> possible.  Some of these are implemented in this series, including:
> 
>  - Cache defragmentation.	   
> 
>     Currently the SLUB allocator is susceptible to internal
>     fragmentation.  This occurs when a large number of cached objects
>     are allocated and then freed in an arbitrary order.  As the cache
>     fragments the number of pages used by the partial slabs list
>     increases.  This wastes memory.
> 
>     Patch set implements the machinery to facilitate conditional cache
>     defragmentation (via kmem_cache_defrag()) and unconditional
>     defragmentation (via kmem_cache_shrink()).  Various sysfs knobs are
>     provided to interact with and configure this.
> 
>     Patch set implements movable objects and cache defragmentation for
>     the XArray.
> 
>  - Moving objects to and from a specific NUMA node.
> 
>  - Balancing objects across all NUMA nodes.
> 
> We add a test module to facilitate playing around with movable objects
> and a python test suite that uses the module.
> 
> Everything except the NUMA stuff was tested on bare metal, the NUMA
> stuff was tested with Qemu NUMA emulation.
> 
> Possible further work:
> 
> 1. Implementing movable objects for the inode and dentry caches.
> 
> 2. Tying into the page migration and page defragmentation logic so that
>    so far unmovable pages that are in the way of creating a contiguous
>    block of memory will become movable.  This would mean checking for
>    slab pages in the migration logic and calling slab to see if it can
>    move the page by migrating all objects.


Hi Tobin!

Very interesting and promising patchset! Looking forward for inode/dentry
moving support, might be a big deal for allocating huge pages dynamically.

Thanks!