linux-kernel - Re: [PATCH v4 10/22] slab: add optimized sheaf refill from partial list

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aXsPoPXvAWoSizCq@hyeyoo>
Date: Thu, 29 Jan 2026 16:43:28 +0900
From: Harry Yoo <harry.yoo@...cle.com>
To: Hao Li <hao.li@...ux.dev>
Cc: Vlastimil Babka <vbabka@...e.cz>, Petr Tesarik <ptesarik@...e.com>,
        Christoph Lameter <cl@...two.org>,
        David Rientjes <rientjes@...gle.com>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Uladzislau Rezki <urezki@...il.com>,
        "Liam R. Howlett" <Liam.Howlett@...cle.com>,
        Suren Baghdasaryan <surenb@...gle.com>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        Alexei Starovoitov <ast@...nel.org>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, linux-rt-devel@...ts.linux.dev,
        bpf@...r.kernel.org, kasan-dev@...glegroups.com
Subject: Re: [PATCH v4 10/22] slab: add optimized sheaf refill from partial
 list

On Mon, Jan 26, 2026 at 03:12:03PM +0800, Hao Li wrote:
> On Fri, Jan 23, 2026 at 07:52:48AM +0100, Vlastimil Babka wrote:
> > At this point we have sheaves enabled for all caches, but their refill
> > is done via __kmem_cache_alloc_bulk() which relies on cpu (partial)
> > slabs - now a redundant caching layer that we are about to remove.
> > 
> > The refill will thus be done from slabs on the node partial list.
> > Introduce new functions that can do that in an optimized way as it's
> > easier than modifying the __kmem_cache_alloc_bulk() call chain.
> > 
> > Introduce struct partial_bulk_context, a variant of struct
> > partial_context that can return a list of slabs from the partial list
> > with the sum of free objects in them within the requested min and max.
> > 
> > Introduce get_partial_node_bulk() that removes the slabs from freelist
> > and returns them in the list. There is a racy read of slab->counters
> > so make sure the non-atomic write in __update_freelist_slow() is not
> > tearing.
> > 
> > Introduce get_freelist_nofreeze() which grabs the freelist without
> > freezing the slab.
> > 
> > Introduce alloc_from_new_slab() which can allocate multiple objects from
> > a newly allocated slab where we don't need to synchronize with freeing.
> > In some aspects it's similar to alloc_single_from_new_slab() but assumes
> > the cache is a non-debug one so it can avoid some actions. It supports
> > the allow_spin parameter, which we always set true here, but the
> > followup change will reuse the function in a context where it may be
> > false.
> > 
> > Introduce __refill_objects() that uses the functions above to fill an
> > array of objects. It has to handle the possibility that the slabs will
> > contain more objects that were requested, due to concurrent freeing of
> > objects to those slabs. When no more slabs on partial lists are
> > available, it will allocate new slabs. It is intended to be only used
> > in context where spinning is allowed, so add a WARN_ON_ONCE check there.
> > 
> > Finally, switch refill_sheaf() to use __refill_objects(). Sheaves are
> > only refilled from contexts that allow spinning, or even blocking.
> > 
> > Reviewed-by: Suren Baghdasaryan <surenb@...gle.com>
> > Signed-off-by: Vlastimil Babka <vbabka@...e.cz>
> > ---
> >  mm/slub.c | 293 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
> >  1 file changed, 272 insertions(+), 21 deletions(-)
> > 
> > diff --git a/mm/slub.c b/mm/slub.c
> > index 22acc249f9c0..142a1099bbc1 100644
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -778,7 +786,8 @@ __update_freelist_slow(struct slab *slab, struct freelist_counters *old,
> >  	slab_lock(slab);
> >  	if (slab->freelist == old->freelist &&
> >  	    slab->counters == old->counters) {
> > -		slab->freelist = new->freelist;
> > +		/* prevent tearing for the read in get_partial_node_bulk() */
> > +		WRITE_ONCE(slab->freelist, new->freelist);
> 
> Should this perhaps be WRITE_ONCE(slab->counters, new->counters) here?

Agreed, this should be WRITE_ONCE(slab->counters, new->counters);

-- 
Cheers,
Harry / Hyeonggon