linux-kernel - Re: [PATCH] mm: clarify COMPACTION Kconfig text

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160825065424.GA4230@dhcp22.suse.cz>
Date:   Thu, 25 Aug 2016 08:54:25 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     David Rientjes <rientjes@...gle.com>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...e.de>, Joonsoo Kim <js1304@...il.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Markus Trippelsdorf <markus@...ppelsdorf.de>,
        linux-mm@...ck.org, LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] mm: clarify COMPACTION Kconfig text

On Wed 24-08-16 17:54:52, David Rientjes wrote:
> On Tue, 23 Aug 2016, Michal Hocko wrote:
> 
> > From: Michal Hocko <mhocko@...e.com>
> > 
> > The current wording of the COMPACTION Kconfig help text doesn't
> > emphasise that disabling COMPACTION might cripple the page allocator
> > which relies on the compaction quite heavily for high order requests and
> > an unexpected OOM can happen with the lack of compaction. Make sure
> > we are vocal about that.
> > 
> 
> Since when has this been an issue? 

Well, pretty much since we have dropped the lumpy reclaim. 

> I don't believe it has been an issue in the past for any archs that
> don't use thp.

Well, fragmentation is a real problem and order-0 reclaim will be never
anywhere close to reliably provide higher order pages. Well, reclaiming
a lot of memory can increase the probability of a success but that
can quite often lead to over reclaim and long stalls. There are other
sources of high order requests than THP so this is not about THP at all
IMHO.

> > Signed-off-by: Michal Hocko <mhocko@...e.com>
> > ---
> >  mm/Kconfig | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index 78a23c5c302d..0dff2f05b6d1 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -262,7 +262,14 @@ config COMPACTION
> >  	select MIGRATION
> >  	depends on MMU
> >  	help
> > -	  Allows the compaction of memory for the allocation of huge pages.
> > +          Compaction is the only memory management component to form
> > +          high order (larger physically contiguous) memory blocks
> > +          reliably. Page allocator relies on the compaction heavily and
> > +          the lack of the feature can lead to unexpected OOM killer
> > +          invocation for high order memory requests. You shouldnm't
> > +          disable this option unless there is really a strong reason for
> > +          it and then we are really interested to hear about that at
> > +          linux-mm@...ck.org.
> >  
> >  #
> >  # support for page migration
> 
> This seems to strongly suggest that all kernels should be built with 
> CONFIG_COMPACTION and its requirement

Yes. Do you see any reason why the compaction should be disabled and we
should rely solely on order-0 reclaim?

> , CONFIG_MIGRATION.  Migration has a 
> dependency of NUMA or memory hot-remove (not all popular).  Compaction can 
> defragment memory within single zone without reliance on NUMA.

I am not sure I am following you here.
MIGRATION depends on (NUMA || ARCH_ENABLE_MEMORY_HOTREMOVE || COMPACTION || CMA) && MMU
 
> This seems like a very bizarre requirement and I'm wondering where we 
> regressed from this thp-only behavior.

-- 
Michal Hocko
SUSE Labs