linux-kernel - Re: [PATCH] mm: Do not stall in synchronous compaction for THP allocations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20111114160345.01e94987.akpm@linux-foundation.org>
Date:	Mon, 14 Nov 2011 16:03:45 -0800
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Mel Gorman <mgorman@...e.de>
Cc:	Minchan Kim <minchan.kim@...il.com>, Jan Kara <jack@...e.cz>,
	Andy Isaacson <adi@...apodia.org>,
	Johannes Weiner <jweiner@...hat.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: [PATCH] mm: Do not stall in synchronous compaction for THP
 allocations

On Fri, 11 Nov 2011 10:01:56 +0000
Mel Gorman <mgorman@...e.de> wrote:

> On Thu, Nov 10, 2011 at 03:12:11PM -0800, Andrew Morton wrote:
> > On Thu, 10 Nov 2011 16:13:31 +0000
> > Mel Gorman <mgorman@...e.de> wrote:
> > 
> > > This patch once again prevents sync migration for transparent
> > > hugepage allocations as it is preferable to fail a THP allocation
> > > than stall.
> > 
> > Who said?  ;)
> 
> Everyone who ever complained about stalls due to writing to USB :)

oh, here it is.  Who broke mesage threading?

> In the last few releases, we have squashed a number of stalls due
> to writing pages from the end of the LRU but sync compaction is a
> relatively recent new root cause of stalls.
> 
> > Presumably some people would prefer to get lots of
> > huge pages for their 1000-hour compute job, and waiting a bit to get
> > those pages is acceptable.
> > 
> 
> A 1000-hour compute job will have its pages collapsed into hugepages by
> khugepaged so they might not have the huge pages at the very beginning
> but they get them. With khugepaged in place, there should be no need for
> an additional tuneable.

OK...

> > Do we have the accounting in place for us to be able to determine how
> > many huge page allocation attempts failed due to this change?
> > 
> 
> thp_fault_fallback is the big one. It is incremented if we fail to
> 	allocate a hugepage during fault in either
> 	do_huge_pmd_anonymous_page or do_huge_pmd_wp_page_fallback
> 
> thp_collapse_alloc_failed is also very interesting. It is incremented
> 	if khugepaged tried to collapse pages into a hugepage and
> 	failed the allocation
> 
> The user has the  option of monitoring their compute jobs hugepage
> usage by reading /proc/PID/smaps and looking at the AnonHugePages
> count for the large mappings of interest.

Fair enough.  One slight problem though:

akpm:/usr/src/25> grep -r thp_collapse_alloc_failed Documentation 
akpm:/usr/src/25> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/