linux-kernel - Re: [PATCH v4 3/3] mm/vmscan: don't demote if there is not enough free memory in the lower memory tier

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aWfXHKDzoQBsHxAp@gourry-fedora-PF4VCD3F>
Date: Wed, 14 Jan 2026 12:49:16 -0500
From: Gregory Price <gourry@...rry.net>
To: Akinobu Mita <akinobu.mita@...il.com>
Cc: Michal Hocko <mhocko@...e.com>, linux-cxl@...r.kernel.org,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	akpm@...ux-foundation.org, axelrasmussen@...gle.com,
	yuanchu@...gle.com, weixugc@...gle.com, hannes@...xchg.org,
	david@...nel.org, zhengqi.arch@...edance.com,
	shakeel.butt@...ux.dev, lorenzo.stoakes@...cle.com,
	Liam.Howlett@...cle.com, vbabka@...e.cz, rppt@...nel.org,
	surenb@...gle.com, ziy@...dia.com, matthew.brost@...el.com,
	joshua.hahnjy@...il.com, rakie.kim@...com, byungchul@...com,
	ying.huang@...ux.alibaba.com, apopple@...dia.com,
	bingjiao@...gle.com, jonathan.cameron@...wei.com,
	pratyush.brahma@....qualcomm.com
Subject: Re: [PATCH v4 3/3] mm/vmscan: don't demote if there is not enough
 free memory in the lower memory tier

On Wed, Jan 14, 2026 at 09:51:28PM +0900, Akinobu Mita wrote:
> can_demote() is called from four places.
> I tried modifying the patch to change the behavior only when can_demote()
> is called from shrink_folio_list(), but the problem was not fixed
> (oom did not occur).
> 
> Similarly, changing the behavior of can_demote() when called from
> can_reclaim_anon_pages(), shrink_folio_list(), and can_age_anon_pages(),
> but not when called from get_swappiness(), did not fix the problem either
> (oom did not occur).
> 
> Conversely, changing the behavior only when called from get_swappiness(),
> but not changing the behavior of can_reclaim_anon_pages(),
> shrink_folio_list(), and can_age_anon_pages(), fixed the problem
> (oom did occur).
> 
> Therefore, it appears that the behavior of get_swappiness() is important
> in this issue.

"It appears that..." and the process of twiddling bits and observing
behavior does not strike confidence in this solution.

Can you take another go at trying to define the bad interaction more
explicitly? I worry that we're modifying vmscan.c behavior to induce an
OOM for a corner case - but it will also cause another regression.

(I am guilty of doing this)

~Gregory