linux-kernel - [PATCH 0/5] Candidate fixes for premature OOM kills with node-lru v2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1469110261-7365-1-git-send-email-mgorman@techsingularity.net>
Date:	Thu, 21 Jul 2016 15:10:56 +0100
From:	Mel Gorman <mgorman@...hsingularity.net>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Johannes Weiner <hannes@...xchg.org>,
	Minchan Kim <minchan@...nel.org>,
	Michal Hocko <mhocko@...e.cz>,
	Vlastimil Babka <vbabka@...e.cz>,
	Linux-MM <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Mel Gorman <mgorman@...hsingularity.net>
Subject: [PATCH 0/5] Candidate fixes for premature OOM kills with node-lru v2

Both Joonsoo Kim and Minchan Kim have reported premature OOM kills.
The common element is a zone-constrained allocation failings. Two factors
appear to be at fault -- pgdat being considered unreclaimable prematurely
and insufficient rotation of the active list.

The series is in three basic parts;

Patches 1-3 add per-zone stats back in. The actual stats patch is different
	to Minchan's as the original patch did not account for unevictable
	LRU which would corrupt counters. The second two patches remove
	approximations based on pgdat statistics. It's effectively a
	revert of "mm, vmstat: remove zone and node double accounting
	by approximating retries" but different LRU stats are used. This
	is better than a full revert or a reworking of the series as it
	preserves history of why the zone stats are necessary.

	If this work out, we may have to leave the double accounting in
	place for now until an alternative cheap solution presents itself.

Patch 4 rotates inactive/active lists for lowmem allocations. This is also
	quite different to Minchan's patch as the original patch did not
	account for memcg and would rotate if *any* eligible zone needed
	rotation which may rotate excessively. The new patch considers the
	ratio for all eligible zones which is more in line with node-lru
	in general.

Patch 5 accounts for skipped pages as partial scanned. This avoids the pgdat
	being prematurely marked unreclaimable while still allowing it to
	be marked unreclaimable if there are no reclaimable pages.

These patches did not OOM for me on a 2G 32-bit KVM instance while running
a stress test for an hour. Preliminary tests on a 64-bit system using a
parallel dd workload did not show anything alarming.

If an OOM is detected then please post the full OOM message.

Optionally please test without patch 5 if an OOM occurs.

 include/linux/mm_inline.h | 19 ++---------
 include/linux/mmzone.h    |  7 ++++
 include/linux/swap.h      |  1 +
 mm/compaction.c           | 20 +----------
 mm/migrate.c              |  2 ++
 mm/page-writeback.c       | 17 +++++-----
 mm/page_alloc.c           | 59 +++++++++++----------------------
 mm/vmscan.c               | 84 ++++++++++++++++++++++++++++++++++++++---------
 mm/vmstat.c               |  6 ++++
 9 files changed, 116 insertions(+), 99 deletions(-)

-- 
2.6.4