linux-kernel - Re: [PATCH 2/2] mm: warn about ALLOC_NO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151126095205.GB7953@dhcp22.suse.cz>
Date:	Thu, 26 Nov 2015 10:52:05 +0100
From:	Michal Hocko <mhocko@...nel.org>
To:	David Rientjes <rientjes@...gle.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Mel Gorman <mgorman@...e.de>,
	Johannes Weiner <hannes@...xchg.org>, linux-mm@...ck.org,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 2/2] mm: warn about ALLOC_NO_WATERMARKS request failures

On Wed 25-11-15 13:01:56, David Rientjes wrote:
> On Wed, 25 Nov 2015, Michal Hocko wrote:
> 
> > > > @@ -2642,6 +2644,13 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
> > > >  	if (zonelist_rescan)
> > > >  		goto zonelist_scan;
> > > >  
> > > > +	/* WARN only once unless min_free_kbytes is updated */
> > > > +	if (warn_alloc_no_wmarks && (alloc_flags & ALLOC_NO_WATERMARKS)) {
> > > > +		warn_alloc_no_wmarks = 0;
> > > > +		WARN(1, "Memory reserves are depleted for order:%d, mode:0x%x."
> > > > +			" You might consider increasing min_free_kbytes\n",
> > > > +			order, gfp_mask);
> > > > +	}
> > > >  	return NULL;
> > > >  }
> > > >  
> > > 
> > > Doesn't this warn for high-order allocations prior to the first call to 
> > > direct compaction whereas min_free_kbytes may be irrelevant?
> > 
> > Hmm, you are concerned about high order ALLOC_NO_WATERMARKS allocation
> > which happen prior to compaction, right? I am wondering whether there
> > are reasonable chances that a compaction would make a difference if we
> > are so depleted that there is no single page with >= order.
> > ALLOC_NO_WATERMARKS with high order allocations should be rare if
> > existing at all.
> > 
> 
> No, I'm concerned about get_page_from_freelist() failing for an order-9 
> allocation due to _fragmentation_ and then emitting this warning although 
> free watermarks may be gigabytes of memory higher than min watermarks.

Hmm, should we allow ALLOC_NO_WATERMARKS for order-9 (or >
PAGE_ALLOC_COSTLY_ORDER for that matter) allocations though?  What would
be the point if they are allowed to fail and so they cannot be relied on
inherently?
I can see that we might do that currently - e.g. TIF_MEMDIE might be
set while doing hugetlb page allocation but I seriously doubt that this is
intentional and probably worth fixing.

> > > Providing 
> > > the order is good, but there's no indication when min_free_kbytes may be 
> > > helpful from this warning. 
> > 
> > I am not sure I understand what you mean here.
> > 
> 
> You show the order of the failed allocation in your new warning.  Good.  
> It won't help to raise min_free_kbytes to infinity if the high-order 
> allocation failed due to fragmentation.  Does that make sense?

Sure this makes sense but as I've tried to argue the warning is just a
hint. It should warn that something unexpected is happening and offer
a workaround. And yes increasing min_free_kbytes helps to keep more
high order pages availble from my experience.
If the workaround doesn't help I suspect the bug report would come more
promptly. Your example about order-9 ALLOC_NO_WATERMARKS failure is more
than exaggarated IMHO.

> > > WARN() isn't even going to show the state of memory.
> > 
> > I was considering to do that but it would make the code unnecessarily
> > more complex. If the allocation is allowed to fail it would dump the
> > allocation failure. The purpose of the message is to tell us that
> > reserves are not sufficient. I am not sure seeing the memory state dump
> > would help us much more.
> > 
> 
> If the purpsoe of the message is to tell us when reserves are 
> insufficient, it doesn't achieve that purpose if allocations fail due to 
> fragmentation or lowmem_reserve_ratio.

Do you have any better suggestion or you just think that warning about
depleted reserves doesn't make any sense at all?

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/