linux-kernel - Re: [PATCH 2/7] mm: introduce kvmalloc and kvmalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 9 Jul 2015 10:45:30 -0400 (EDT)
From:	Mikulas Patocka <mpatocka@...hat.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
cc:	Mike Snitzer <msnitzer@...hat.com>,
	"Alasdair G. Kergon" <agk@...hat.com>,
	Edward Thornber <thornber@...hat.com>,
	David Rientjes <rientjes@...gle.com>,
	Vivek Goyal <vgoyal@...hat.com>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, dm-devel@...hat.com,
	Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [PATCH 2/7] mm: introduce kvmalloc and kvmalloc_node



On Wed, 8 Jul 2015, Andrew Morton wrote:

> On Wed, 8 Jul 2015 19:03:08 -0400 (EDT) Mikulas Patocka <mpatocka@...hat.com> wrote:
> 
> > 
> > 
> > On Tue, 7 Jul 2015, Andrew Morton wrote:
> > 
> > > On Tue, 7 Jul 2015 11:10:09 -0400 (EDT) Mikulas Patocka <mpatocka@...hat.com> wrote:
> > > 
> > > > Introduce the functions kvmalloc and kvmalloc_node. These functions
> > > > provide reliable allocation of object of arbitrary size. They attempt to
> > > > do allocation with kmalloc and if it fails, use vmalloc. Memory allocated
> > > > with these functions should be freed with kvfree.
> > > 
> > > Sigh.  We've resisted doing this because vmalloc() is somewhat of a bad
> > > thing, and we don't want to make it easy for people to do bad things.
> > > 
> > > And vmalloc is bad because a) it's slow and b) it does GFP_KERNEL
> > > allocations for page tables and c) it is susceptible to arena
> > > fragmentation.
> > 
> > This patch makes less use of vmalloc.
> > 
> > The typical pattern is that someone notices random failures due to memory 
> > fragmentation in some subsystem that uses large kmalloc - so he replaces 
> > kmalloc with vmalloc - and the code gets slower because of that. With this 
> > patch, you can replace many vmalloc users with kvmalloc - and vmalloc will 
> > be used only very rarely, when the memory is too fragmented for kmalloc.
> 
> Yes, I guess there is that.
> 
> > Here I'm sending next version of the patch with comments added.
> 
> You didn't like kvzalloc()?  We can always add those later...
> 
> > --- linux-4.2-rc1.orig/include/linux/mm.h	2015-07-07 15:58:11.000000000 +0200
> > +++ linux-4.2-rc1/include/linux/mm.h	2015-07-08 19:22:24.000000000 +0200
> > @@ -400,6 +400,11 @@ static inline int is_vmalloc_or_module_a
> >  }
> >  #endif
> >  
> > +extern void *kvmalloc_node(size_t size, gfp_t gfp, int node);
> > +static inline void *kvmalloc(size_t size, gfp_t gfp)
> > +{
> > +	return kvmalloc_node(size, gfp, NUMA_NO_NODE);
> > +}
> >  extern void kvfree(const void *addr);
> >  
> >  static inline void compound_lock(struct page *page)
> > Index: linux-4.2-rc1/mm/util.c
> > ===================================================================
> > --- linux-4.2-rc1.orig/mm/util.c	2015-07-07 15:58:11.000000000 +0200
> > +++ linux-4.2-rc1/mm/util.c	2015-07-08 19:22:26.000000000 +0200
> > @@ -316,6 +316,61 @@ unsigned long vm_mmap(struct file *file,
> >  }
> >  EXPORT_SYMBOL(vm_mmap);
> >  
> > +void *kvmalloc_node(size_t size, gfp_t gfp, int node)
> > +{
> > +	void *p;
> > +	unsigned uninitialized_var(noio_flag);
> > +
> > +	/* vmalloc doesn't support no-wait allocations */
> > +	WARN_ON_ONCE(!(gfp & __GFP_WAIT));
> > +
> > +	if (likely(size <= KMALLOC_MAX_SIZE)) {
> > +		/*
> > +		 * Use __GFP_NORETRY so that we don't loop waiting for the
> > +		 *	allocation - we don't have to loop here, if the memory
> > +		 *	is too fragmented, we fallback to vmalloc.
> 
> I'm not sure about this decision.  The direct reclaim retry code is the
> normal default behaviour and becomes more important with larger allocation
> attempts.  So why turn it off, and make it more likely that we return
> vmalloc memory?

It can avoid triggering the OOM killer in case of fragmented memory.

This is general question - if the code can handle allocation failure 
gracefully, what gfp flags should it use? Maybe add some flag 
__GFP_MAYFAIL instead of __GFP_NORETRY that changes the behavior in 
desired way?

Mikulas
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/