linux-kernel - Re: [BUG] __GFP_THISNODE is not always honored

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1219083412.29323.36.camel@localhost.localdomain>
Date:	Mon, 18 Aug 2008 13:16:52 -0500
From:	Adam Litke <agl@...ibm.com>
To:	Mel Gorman <mel@....ul.ie>
Cc:	linux-mm <linux-mm@...ck.org>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	nacc <nacc@...ux.vnet.ibm.com>, apw <apw@...dowen.org>,
	agl <agl@...ux.vnet.ibm.com>
Subject: Re: [BUG] __GFP_THISNODE is not always honored

On Mon, 2008-08-18 at 11:59 +0100, Mel Gorman wrote:
> On (15/08/08 17:01), Adam Litke didst pronounce:
> > While running the libhugetlbfs test suite on a NUMA machine with 2.6.27-rc3, I
> > discovered some strange behavior with __GFP_THISNODE.  The hugetlb function
> > alloc_fresh_huge_page_node() calls alloc_pages_node() with __GFP_THISNODE but
> > occasionally a page that is not on the requested node is returned. 
> 
> That's bad in itself and has wider reaching consequences than hugetlb
> getting its counters wrong. I believe SLUB depends on __GFP_THISNODE
> being obeyed for example. Can you boot the machine in question with
> mminit_loglevel=4 and loglevel=8 set on the command line and send me the
> dmesg please? It should output the zonelists and I might be able to
> figure out what's going wrong. Thanks

dmesg output is attached.

> > Since the
> > hugetlb code assumes that the page will be on the requested node, badness follows
> > when the page is added to the wrong node's free_list.
> > 
> > There is clearly something wrong with the buddy allocator since __GFP_THISNODE
> > cannot be trusted.  Until that is fixed, the hugetlb code should not assume
> > that the newly allocated page is on the node asked for.  This patch prevents
> > the hugetlb pool counters from being corrupted and allows the code to cope with
> > unbalanced numa allocations.
> > 
> > So far my debugging has led me to get_page_from_freelist() inside the
> > for_each_zone_zonelist() loop.  When buffered_rmqueue() returns a page I
> > compare the value of page_to_nid(page), zone->node and the node that the
> > hugetlb code requested with __GFP_THISNODE.  These all match -- except when the
> > problem triggers.  In that case, zone->node matches the node we asked for but
> > page_to_nid() does not.
> > 
> 
> Feels like the wrong zonelist is being used. The dmesg with
> mminit_loglevel may tell.
> 
> > Workaround patch:
> > diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> > index 67a7119..7a30a61 100644
> > --- a/mm/hugetlb.c
> > +++ b/mm/hugetlb.c
> > @@ -568,7 +568,7 @@ static struct page *alloc_fresh_huge_page_node(struct hstate *h, int nid)
> >  			__free_pages(page, huge_page_order(h));
> >  			return NULL;
> >  		}
> > -		prep_new_huge_page(h, page, nid);
> > +		prep_new_huge_page(h, page, page_to_nid(page));
> >  	}
> 
> This will mask the bug for hugetlb but I wonder if this should be a
> VM_BUG_ON(page_to_nid(page) != nid) ?

Yeah, the patch was provided for illustrative purposes only.

-- 
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center

View attachment "dmesg.out" of type "text/plain" (39045 bytes)