lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 25 Aug 2016 08:54:25 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     David Rientjes <rientjes@...gle.com>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...e.de>, Joonsoo Kim <js1304@...il.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Markus Trippelsdorf <markus@...ppelsdorf.de>,
        linux-mm@...ck.org, LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] mm: clarify COMPACTION Kconfig text

On Wed 24-08-16 17:54:52, David Rientjes wrote:
> On Tue, 23 Aug 2016, Michal Hocko wrote:
> 
> > From: Michal Hocko <mhocko@...e.com>
> > 
> > The current wording of the COMPACTION Kconfig help text doesn't
> > emphasise that disabling COMPACTION might cripple the page allocator
> > which relies on the compaction quite heavily for high order requests and
> > an unexpected OOM can happen with the lack of compaction. Make sure
> > we are vocal about that.
> > 
> 
> Since when has this been an issue? 

Well, pretty much since we have dropped the lumpy reclaim. 

> I don't believe it has been an issue in the past for any archs that
> don't use thp.

Well, fragmentation is a real problem and order-0 reclaim will be never
anywhere close to reliably provide higher order pages. Well, reclaiming
a lot of memory can increase the probability of a success but that
can quite often lead to over reclaim and long stalls. There are other
sources of high order requests than THP so this is not about THP at all
IMHO.

> > Signed-off-by: Michal Hocko <mhocko@...e.com>
> > ---
> >  mm/Kconfig | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index 78a23c5c302d..0dff2f05b6d1 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -262,7 +262,14 @@ config COMPACTION
> >  	select MIGRATION
> >  	depends on MMU
> >  	help
> > -	  Allows the compaction of memory for the allocation of huge pages.
> > +          Compaction is the only memory management component to form
> > +          high order (larger physically contiguous) memory blocks
> > +          reliably. Page allocator relies on the compaction heavily and
> > +          the lack of the feature can lead to unexpected OOM killer
> > +          invocation for high order memory requests. You shouldnm't
> > +          disable this option unless there is really a strong reason for
> > +          it and then we are really interested to hear about that at
> > +          linux-mm@...ck.org.
> >  
> >  #
> >  # support for page migration
> 
> This seems to strongly suggest that all kernels should be built with 
> CONFIG_COMPACTION and its requirement

Yes. Do you see any reason why the compaction should be disabled and we
should rely solely on order-0 reclaim?

> , CONFIG_MIGRATION.  Migration has a 
> dependency of NUMA or memory hot-remove (not all popular).  Compaction can 
> defragment memory within single zone without reliance on NUMA.

I am not sure I am following you here.
MIGRATION depends on (NUMA || ARCH_ENABLE_MEMORY_HOTREMOVE || COMPACTION || CMA) && MMU
 
> This seems like a very bizarre requirement and I'm wondering where we 
> regressed from this thp-only behavior.

-- 
Michal Hocko
SUSE Labs

Powered by blists - more mailing lists