linux-kernel - Re: [PATCH 0/3] OOM detection rework v4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 8 Jan 2016 13:37:44 +0100
From:	Michal Hocko <mhocko@...nel.org>
To:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
Cc:	akpm@...ux-foundation.org, torvalds@...ux-foundation.org,
	hannes@...xchg.org, mgorman@...e.de, rientjes@...gle.com,
	hillf.zj@...baba-inc.com, kamezawa.hiroyu@...fujitsu.com,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 0/3] OOM detection rework v4

On Mon 28-12-15 23:13:31, Tetsuo Handa wrote:
> Tetsuo Handa wrote:
> > Tetsuo Handa wrote:
> > > I got OOM killers while running heavy disk I/O (extracting kernel source,
> > > running lxr's genxref command). (Environ: 4 CPUs / 2048MB RAM / no swap / XFS)
> > > Do you think these OOM killers reasonable? Too weak against fragmentation?
> >
> > Since I cannot establish workload that caused December 24's natural OOM
> > killers, I used the following stressor for generating similar situation.
> >
> 
> I came to feel that I am observing a different problem which is currently
> hidden behind the "too small to fail" memory-allocation rule. That is, tasks
> requesting order > 0 pages are continuously losing the competition when
> tasks requesting order = 0 pages dominate, for reclaimed pages are stolen
> by tasks requesting order = 0 pages before reclaimed pages are combined to
> order > 0 pages (or maybe order > 0 pages are immediately split into
> order = 0 pages due to tasks requesting order = 0 pages).
> 
> Currently, order <= PAGE_ALLOC_COSTLY_ORDER allocations implicitly retry
> unless chosen by the OOM killer. Therefore, even if tasks requesting
> order = 2 pages lost the competition when there are tasks requesting
> order = 0 pages, the order = 2 allocation request is implicitly retried
> and therefore the OOM killer is not invoked (though there is a problem that
> tasks requesting order > 0 allocation will stall as long as tasks requesting
> order = 0 pages dominate).

Yes this is possible and nothing new. High order allocations (even small
orders) are never for free and more expensive than order-0. I have seen
an OOM killer striking while there were megs of free memory on a larger
machine just because of the high fragmentation.

> But this patchset introduced a limit of 16 retries.

We retry 16 times _only_ if the reclaim hasn't made _any_ progress
which means it hasn't reclaimed a single page. We can still fail due to
watermarks check for the required order but I think this is a correct
and desirable behavior because there is no guarantee that lower order
pages will get coalesced after more retries. The primary point of this
rework is to make the whole thing more deterministic.

So we can see some OOM reports for high orders (<COSTLY) which would
survive before just because we have retried so many times that we
end up allocating that single high order page but this was a pure luck
and indeterministic behavior. That being said I agree we might end up
doing some more tuning for non-costly high order allocation but it
should be bounded as well and based on failures on some reasonable
workloads. I haven't got to OOM reports you have posted yet but I
definitely plan to check them soon.

[...]
-- 
Michal Hocko
SUSE Labs