lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Wed, 06 May 2015 13:58:45 -0400 From: Waiman Long <waiman.long@...com> To: Mel Gorman <mgorman@...e.de> CC: Andrew Morton <akpm@...ux-foundation.org>, Nathan Zimmer <nzimmer@....com>, Dave Hansen <dave.hansen@...el.com>, Scott Norton <scott.norton@...com>, Daniel J Blueman <daniel@...ascale.com>, Linux-MM <linux-mm@...ck.org>, LKML <linux-kernel@...r.kernel.org> Subject: Re: [PATCH 0/13] Parallel struct page initialisation v4 On 05/06/2015 06:22 AM, Mel Gorman wrote: > On Wed, May 06, 2015 at 08:12:46AM +0100, Mel Gorman wrote: >> On Tue, May 05, 2015 at 03:25:49PM -0700, Andrew Morton wrote: >>> On Tue, 5 May 2015 23:13:29 +0100 Mel Gorman<mgorman@...e.de> wrote: >>> >>>>> Alternatively, the page allocator can go off and synchronously >>>>> initialize some pageframes itself. Keep doing that until the >>>>> allocation attempt succeeds. >>>>> >>>> That was rejected during review of earlier attempts at this feature on >>>> the grounds that it impacted allocator fast paths. >>> eh? Changes are only needed on the allocation-attempt-failed path, >>> which is slow-path. >> We'd have to distinguish between falling back to other zones because the >> high zone is artifically exhausted and normal ALLOC_BATCH exhaustion. We'd >> also have to avoid falling back to remote nodes prematurely. While I have >> not tried an implementation, I expected they would need to be in the fast >> paths unless I used jump labels to get around it. I'm going to try altering >> when we initialise instead so that it happens earlier. >> > Which looks as follows. Waiman, a test on the 24TB machine would be > appreciated again. This patch should be applied instead of "mm: meminit: > Take into account that large system caches scale linearly with memory" > > ---8<--- > mm: meminit: Finish initialisation of memory before basic setup > > Waiman Long reported that 24TB machines hit OOM during basic setup when > struct page initialisation was deferred. One approach is to initialise memory > on demand but it interferes with page allocator paths. This patch creates > dedicated threads to initialise memory before basic setup. It then blocks > on a rw_semaphore until completion as a wait_queue and counter is overkill. > This may be slower to boot but it's simplier overall and also gets rid of a > lot of section mangling which existed so kswapd could do the initialisation. > > Signed-off-by: Mel Gorman<mgorman@...e.de> > This patch moves the deferred meminit from kswapd to its own kernel threads started after smp_init(). However, the hash table allocation was done earlier than that. It seems like it will still run out of memory in the 24TB machine that I tested on. I will certainly try it out, but I doubt it will solve the problem on its own. Cheers, Longman -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists