linux-kernel - Re: [PATCH 8/9] mm: vmalloc: Offload free_vmap_area

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 6 Jun 2023 20:11:04 +0800
From:   Baoquan He <bhe@...hat.com>
To:     Uladzislau Rezki <urezki@...il.com>
Cc:     linux-mm@...ck.org, Andrew Morton <akpm@...ux-foundation.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Lorenzo Stoakes <lstoakes@...il.com>,
        Christoph Hellwig <hch@...radead.org>,
        Matthew Wilcox <willy@...radead.org>,
        "Liam R . Howlett" <Liam.Howlett@...cle.com>,
        Dave Chinner <david@...morbit.com>,
        "Paul E . McKenney" <paulmck@...nel.org>,
        Joel Fernandes <joel@...lfernandes.org>,
        Oleksiy Avramchenko <oleksiy.avramchenko@...y.com>
Subject: Re: [PATCH 8/9] mm: vmalloc: Offload free_vmap_area_lock global lock

On 06/06/23 at 11:01am, Uladzislau Rezki wrote:
> On Mon, Jun 05, 2023 at 08:43:39AM +0800, Baoquan He wrote:
> > On 05/22/23 at 01:08pm, Uladzislau Rezki (Sony) wrote:
> > ......  
> > > +static unsigned long
> > > +this_cpu_zone_alloc_fill(struct cpu_vmap_zone *z,
> > > +	unsigned long size, unsigned long align,
> > > +	gfp_t gfp_mask, int node)
> > > +{
> > > +	unsigned long addr = VMALLOC_END;
> > > +	struct vmap_area *va;
> > > +
> > > +	/*
> > > +	 * It still can race. One task sets a progress to
> > > +	 * 1 a second one gets preempted on entry, the first
> > > +	 * zeroed the progress flag and second proceed with
> > > +	 * an extra prefetch.
> > > +	 */
> > > +	if (atomic_xchg(&z->fill_in_progress, 1))
> > > +		return addr;
> > > +
> > > +	va = kmem_cache_alloc_node(vmap_area_cachep, gfp_mask, node);
> > > +	if (unlikely(!va))
> > > +		goto out;
> > > +
> > > +	spin_lock(&free_vmap_area_lock);
> > > +	addr = __alloc_vmap_area(&free_vmap_area_root, &free_vmap_area_list,
> > > +		cvz_size, 1, VMALLOC_START, VMALLOC_END);
> > > +	spin_unlock(&free_vmap_area_lock);
> > 
> > The 'z' is passed in from this_cpu_zone_alloc(), and it's got with
> > raw_cpu_ptr(&cpu_vmap_zone). Here when we try to get chunk of cvz_size
> > from free_vmap_area_root/free_vmap_area_list, how can we guarantee it
> > must belong to the 'z' zone? With my understanding, __alloc_vmap_area()
> > will get efficient address range sequentially bottom up from
> > free_vmap_area_root. Please correct me if I am wrong.
> > 
> We do not guarantee that and it does not worth it. The most important is:
> 
> If we search a zone that exactly match a CPU-id the usage of a global
> vmap space becomes more wider, i.e. toward a high address space. This is
> not good because we can affect other users which allocate within a specific
> range. On a big system it might be a problem. Therefore a pre-fetch is done 
> sequentially on demand.
> 
> Secondly, i do not see much difference in performance if we follow
> exactly CPU-zone-id.

Ah, I see, the allocated range will be put into appropriate zone's
busy tree by calculating its zone via addr_to_cvz(va->va_start). The
cvz->free tree is only a percpu pre-fetch cache. This is smart, thanks a
lot for explanation. 

> 
> > static unsigned long
> > this_cpu_zone_alloc(unsigned long size, unsigned long align, gfp_t gfp_mask, int node)
> > {
> >         struct cpu_vmap_zone *z = raw_cpu_ptr(&cpu_vmap_zone);
> > 	......
> > 	if (addr == VMALLOC_END && left < 4 * PAGE_SIZE)
> >                 addr = this_cpu_zone_alloc_fill(z, size, align, gfp_mask, node);
> > }
> > 
> > > +
> > > +	if (addr == VMALLOC_END) {
> > > +		kmem_cache_free(vmap_area_cachep, va);
> > > +		goto out;
> > > +	}
> > > +
> > > +	va->va_start = addr;
> > > +	va->va_end = addr + cvz_size;
> > > +
> > > +	fbl_lock(z, FREE);
> > > +	va = merge_or_add_vmap_area_augment(va,
> > > +		&fbl_root(z, FREE), &fbl_head(z, FREE));
> > > +	addr = va_alloc(va, &fbl_root(z, FREE), &fbl_head(z, FREE),
> > > +		size, align, VMALLOC_START, VMALLOC_END);
> > > +	fbl_unlock(z, FREE);
> > > +
> > > +out:
> > > +	atomic_set(&z->fill_in_progress, 0);
> > > +	return addr;
> > > +}
> > > +
> > > +static unsigned long
> > > +this_cpu_zone_alloc(unsigned long size, unsigned long align, gfp_t gfp_mask, int node)
> > > +{
> > > +	struct cpu_vmap_zone *z = raw_cpu_ptr(&cpu_vmap_zone);
> > > +	unsigned long extra = align > PAGE_SIZE ? align : 0;
> > > +	unsigned long addr = VMALLOC_END, left = 0;
> > > +
> > > +	/*
> > > +	 * It is disabled, fallback to a global heap.
> > > +	 */
> > > +	if (cvz_size == ULONG_MAX)
> > > +		return addr;
> > > +
> > > +	/*
> > > +	 * Any allocation bigger/equal than one half of
> >                           ~~~~~~typo~~~~~~  bigger than/equal to
> I will rework it!
> 
> --
> Uladzislau Rezki
>