[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAB=+i9T8cOLQt4YprvUghwWZx1nOaiQ-0vV1N1zOOHWAFXza0Q@mail.gmail.com>
Date: Sun, 6 Oct 2024 23:37:22 +0900
From: Hyeonggon Yoo <42.hyeyoo@...il.com>
To: cl@...two.org
Cc: Vlastimil Babka <vbabka@...e.cz>, Pekka Enberg <penberg@...nel.org>,
David Rientjes <rientjes@...gle.com>, Joonsoo Kim <iamjoonsoo.kim@....com>,
Andrew Morton <akpm@...ux-foundation.org>, Roman Gushchin <roman.gushchin@...ux.dev>,
Yang Shi <shy828301@...il.com>, Christoph Lameter <cl@...ux.com>, linux-mm@...ck.org,
linux-kernel@...r.kernel.org, linux-arm-kernel@...ts.infradead.org,
Huang Shijie <shijie@...amperecomputing.com>
Subject: Re: [PATCH v3] SLUB: Add support for per object memory policies
On Wed, Oct 2, 2024 at 4:08 AM Christoph Lameter via B4 Relay
<devnull+cl.gentwo.org@...nel.org> wrote:
>
> From: Christoph Lameter <cl@...two.org>
>
> The old SLAB allocator used to support memory policies on a per
> allocation bases. In SLUB the memory policies are applied on a
> per page frame / folio bases. Doing so avoids having to check memory
> policies in critical code paths for kmalloc and friends.
>
> This worked on general well on Intel/AMD/PowerPC because the
> interconnect technology is mature and can minimize the latencies
> through intelligent caching even if a small object is not
> placed optimally.
>
> However, on ARM we have an emergence of new NUMA interconnect
> technology based more on embedded devices. Caching of remote content
> can currently be ineffective using the standard building blocks / mesh
> available on that platform. Such architectures benefit if each slab
> object is individually placed according to memory policies
> and other restrictions.
>
> This patch adds another kernel parameter
>
> slab_strict_numa
>
> If that is set then a static branch is activated that will cause
> the hotpaths of the allocator to evaluate the current memory
> allocation policy. Each object will be properly placed by
> paying the price of extra processing and SLUB will no longer
> defer to the page allocator to apply memory policies at the
> folio level.
>
> This patch improves performance of memcached running
> on Ampere Altra 2P system (ARM Neoverse N1 processor)
> by 3.6% due to accurate placement of small kernel objects.
>
> Tested-by: Huang Shijie <shijie@...amperecomputing.com>
> Signed-off-by: Christoph Lameter (Ampere) <cl@...two.org>
> ---
> Changes in v3:
> - Make the static key a static in slub.c
> - Use pr_warn / pr_info instead of printk
> - Link to v2: https://lore.kernel.org/r/20240906-strict_numa-v2-1-f104e6de6d1e@gentwo.org
>
> Changes in v2:
> - Fix various issues
> - Testing
> ---
> mm/slub.c | 42 ++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 42 insertions(+)
>
> diff --git a/mm/slub.c b/mm/slub.c
> index 21f71cb6cc06..7ae94f79740d 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -218,6 +218,10 @@ DEFINE_STATIC_KEY_FALSE(slub_debug_enabled);
> #endif
> #endif /* CONFIG_SLUB_DEBUG */
>
> +#ifdef CONFIG_NUMA
> +static DEFINE_STATIC_KEY_FALSE(strict_numa);
> +#endif
> +
> /* Structure holding parameters for get_partial() call chain */
> struct partial_context {
> gfp_t flags;
> @@ -3957,6 +3961,28 @@ static __always_inline void *__slab_alloc_node(struct kmem_cache *s,
> object = c->freelist;
> slab = c->slab;
>
> +#ifdef CONFIG_NUMA
> + if (static_branch_unlikely(&strict_numa) &&
> + node == NUMA_NO_NODE) {
> +
> + struct mempolicy *mpol = current->mempolicy;
> +
> + if (mpol) {
> + /*
> + * Special BIND rule support. If existing slab
> + * is in permitted set then do not redirect
> + * to a particular node.
> + * Otherwise we apply the memory policy to get
> + * the node we need to allocate on.
> + */
> + if (mpol->mode != MPOL_BIND || !slab ||
> + !node_isset(slab_nid(slab), mpol->nodes))
> +
> + node = mempolicy_slab_node();
> + }
Is it intentional to allow the local node only (via
mempolicy_slab_node()) in interrupt contexts?
> + }
> +#endif
> +
> if (!USE_LOCKLESS_FAST_PATH() ||
> unlikely(!object || !slab || !node_match(slab, node))) {
> object = __slab_alloc(s, gfpflags, node, addr, c, orig_size);
> @@ -5601,6 +5627,22 @@ static int __init setup_slub_min_objects(char *str)
> __setup("slab_min_objects=", setup_slub_min_objects);
> __setup_param("slub_min_objects=", slub_min_objects, setup_slub_min_objects, 0);
>
> +#ifdef CONFIG_NUMA
> +static int __init setup_slab_strict_numa(char *str)
> +{
> + if (nr_node_ids > 1) {
> + static_branch_enable(&strict_numa);
> + pr_info("SLUB: Strict NUMA enabled.\n");
> + } else
> + pr_warn("slab_strict_numa parameter set on non NUMA system.\n");
nit: this statement should be enclosed within braces per coding style guideline.
Otherwise everything looks good to me (including the document amended).
Best,
Hyeonggon
Powered by blists - more mailing lists