linux-kernel - Re: Newbie's question: memory allocation when reclaiming memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Thu, 5 Nov 2015 09:46:47 +0100
From:	Vlastimil Babka <vbabka@...e.cz>
To:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
	mhocko@...nel.org
Cc:	rientjes@...gle.com, oleg@...hat.com,
	torvalds@...ux-foundation.org, kwalker@...hat.com, cl@...ux.com,
	akpm@...ux-foundation.org, hannes@...xchg.org,
	vdavydov@...allels.com, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, skozina@...hat.com
Subject: Re: Newbie's question: memory allocation when reclaiming memory

On 10/26/2015 12:44 PM, Tetsuo Handa wrote:
> May I ask a newbie question? Say, there is some amount of memory pages
> which can be reclaimed if they are flushed to storage. And lower layer
> might issue memory allocation request in a way which won't cause reclaim
> deadlock (e.g. using GFP_NOFS or GFP_NOIO) when flushing to storage,
> isn't it?
> 
> What I'm worrying is a dependency that __GFP_FS allocation requests think
> that there are reclaimable pages and therefore there is no need to call
> out_of_memory(); and GFP_NOFS allocation requests which the __GFP_FS
> allocation requests depend on (in order to flush to storage) is waiting
> for GFP_NOIO allocation requests; and the GFP_NOIO allocation requests
> which the GFP_NOFS allocation requests depend on (in order to flush to
> storage) are waiting for memory pages to be reclaimed without calling
> out_of_memory(); because gfp_to_alloc_flags() does not favor GFP_NOIO over
> GFP_NOFS nor GFP_NOFS over __GFP_FS which will throttle all allocations
> at the same watermark level.
> 
> How do we guarantee that GFP_NOFS/GFP_NOIO allocations make forward
> progress? What mechanism guarantees that memory pages which __GFP_FS
> allocation requests are waiting for are reclaimed? I assume that there
> is some mechanism; otherwise we can hit silent livelock, can't we?

I've never studied the code myself, but IIRC in all the debates LSF/MM I've
heard it said that GFP_NOIO allocations have mempools that guarantee forward
progress, so when they allocate from this mempool, there should be nothing else
to block the request other than waiting for the actual hardware to finish the
I/O request, and then the memory is returned to mempool and another request can
use it. So there shouldn't be waiting for reclaim at that level, breaking the
livelock you described?

> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@...ck.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@...ck.org"> email@...ck.org </a>
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/