linux-kernel - Re: [PATCH 2/2] mm: warn about ALLOC_NO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.10.1511301418040.10460@chino.kir.corp.google.com>
Date:	Mon, 30 Nov 2015 14:24:31 -0800 (PST)
From:	David Rientjes <rientjes@...gle.com>
To:	Michal Hocko <mhocko@...nel.org>
cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Mel Gorman <mgorman@...e.de>,
	Johannes Weiner <hannes@...xchg.org>, linux-mm@...ck.org,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 2/2] mm: warn about ALLOC_NO_WATERMARKS request
 failures

On Thu, 26 Nov 2015, Michal Hocko wrote:

> > > > > @@ -2642,6 +2644,13 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
> > > > >  	if (zonelist_rescan)
> > > > >  		goto zonelist_scan;
> > > > >  
> > > > > +	/* WARN only once unless min_free_kbytes is updated */
> > > > > +	if (warn_alloc_no_wmarks && (alloc_flags & ALLOC_NO_WATERMARKS)) {
> > > > > +		warn_alloc_no_wmarks = 0;
> > > > > +		WARN(1, "Memory reserves are depleted for order:%d, mode:0x%x."
> > > > > +			" You might consider increasing min_free_kbytes\n",
> > > > > +			order, gfp_mask);
> > > > > +	}
> > > > >  	return NULL;
> > > > >  }
> > > > >  
> > > > 
> > > > Doesn't this warn for high-order allocations prior to the first call to 
> > > > direct compaction whereas min_free_kbytes may be irrelevant?
> > > 
> > > Hmm, you are concerned about high order ALLOC_NO_WATERMARKS allocation
> > > which happen prior to compaction, right? I am wondering whether there
> > > are reasonable chances that a compaction would make a difference if we
> > > are so depleted that there is no single page with >= order.
> > > ALLOC_NO_WATERMARKS with high order allocations should be rare if
> > > existing at all.
> > > 
> > 
> > No, I'm concerned about get_page_from_freelist() failing for an order-9 
> > allocation due to _fragmentation_ and then emitting this warning although 
> > free watermarks may be gigabytes of memory higher than min watermarks.
> 
> Hmm, should we allow ALLOC_NO_WATERMARKS for order-9 (or >
> PAGE_ALLOC_COSTLY_ORDER for that matter) allocations though?  What would
> be the point if they are allowed to fail and so they cannot be relied on
> inherently?

This patch isn't addressing what orders the page allocator allows access 
to memory reserves for, I'm not sure this has anything to do with the 
warning you propose to add.

My concern is that this will start doing

	Memory reserves are depleted for order:9. You might consider increasing min_free_kbytes

in the kernel log with a long stack trace that is going to grab attention 
and then some user will actually follow the advice and see that the 
warning persists because the failure was due to fragmentation rather than 
watermarks.  It would be much better if the warning were only emitted when 
the _watermark_, not fragmentation, was the source of the failure.  That 
is very easy to do, by calling __zone_watermark_ok() for order 0.

I would also suggest that this is done in the same way that GFP_ATOMIC 
allocations fail that have depleted ALLOC_HARD and ALLOC_HARDER memory 
reserves, with something resembling a page allocation failure warning that 
actually presents useful data.  Your patch is already insufficient because 
it doesn't handle __GFP_NOWARN.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/