linux-kernel - Re: 2.6.22-stable causes oomkiller to be invoked

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Pine.LNX.4.64.0712261258050.16862@schroedinger.engr.sgi.com>
Date:	Wed, 26 Dec 2007 13:01:49 -0800 (PST)
From:	Christoph Lameter <clameter@....com>
To:	Dhaval Giani <dhaval@...ux.vnet.ibm.com>
cc:	Andrew Morton <akpm@...ux-foundation.org>, htejun@...il.com,
	gregkh@...e.de, Srivatsa Vaddagiri <vatsa@...ux.vnet.ibm.com>,
	Balbir Singh <balbir@...ibm.com>, maneesh@...ux.vnet.ibm.com,
	lkml <linux-kernel@...r.kernel.org>, stable@...nel.org,
	linux-mm@...ck.org
Subject: Re: 2.6.22-stable causes oomkiller to be invoked

On Fri, 21 Dec 2007, Dhaval Giani wrote:

> No, it does not stop the oom I am seeing here.

Duh. Disregard that patch. It looks like check_pgt_cache() is not called. 
This could happen if tlb_flush_mmu is never called during the 
fork/terminate sequences in your script. pgd_free is called *after* a 
possible tlb flush so the pgd page is on the quicklist (which is good for 
the next process which needs a pgd). The tlb_flush_mmu's during pte 
eviction should trim the quicklist. For some reason this is not happening 
on your box (it works here).

Could you try this script that insures that check_pgt_cache is called 
after every pgd_free?

Index: linux-2.6/arch/x86/mm/pgtable_32.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:10.000000000 -0800
+++ linux-2.6/arch/x86/mm/pgtable_32.c	2007-12-26 12:55:54.000000000 -0800
@@ -366,6 +366,15 @@ void pgd_free(pgd_t *pgd)
 		}
 	/* in the non-PAE case, free_pgtables() clears user pgd entries */
 	quicklist_free(0, pgd_dtor, pgd);
+
+	/*
+	 * We must call check_pgd_cache() here because the pgd is freed after
+	 * tlb flushing and the call to check_pgd_cache. In some cases the VM
+	 * may not call tlb_flush_mmu during process termination (??).
+	 * If this is repeated then we may never call check_pgd_cache.
+	 * The quicklist will grow and grow. So call check_pgd_cache here.
+	 */
+	check_pgt_cache();
 }
 
 void check_pgt_cache(void)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/