linux-kernel - Re: [PATCH v2] mm,oom: Exclude TIF_MEMDIE processes from candidates.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <201601150721.HJC60832.LMOtQHFFFSJOOV@I-love.SAKURA.ne.jp>
Date:	Fri, 15 Jan 2016 07:21:17 +0900
From:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:	rientjes@...gle.com
Cc:	mhocko@...nel.org, hannes@...xchg.org, akpm@...ux-foundation.org,
	mgorman@...e.de, torvalds@...ux-foundation.org, oleg@...hat.com,
	hughd@...gle.com, andrea@...nel.org, riel@...hat.com,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2] mm,oom: Exclude TIF_MEMDIE processes from candidates.

David Rientjes wrote:
> On Thu, 14 Jan 2016, Tetsuo Handa wrote:
> 
> > > > > > @@ -171,7 +195,7 @@ unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg,
> > > > > >  	if (oom_unkillable_task(p, memcg, nodemask))
> > > > > >  		return 0;
> > > > > > 
> > > > > > -	p = find_lock_task_mm(p);
> > > > > > +	p = find_lock_non_victim_task_mm(p);
> > > > > >  	if (!p)
> > > > > >  		return 0;
> > > > > > 
> > > > > 
> > > > > I understand how this may make your test case pass, but I simply don't 
> > > > > understand how this could possibly be the correct thing to do.  This would 
> > > > > cause oom_badness() to return 0 for any process where a thread has 
> > > > > TIF_MEMDIE set.  If the oom killer is called from the page allocator, 
> > > > > kills a thread, and it is recalled before that thread may exit, then this 
> > > > > will panic the system if there are no other eligible processes to kill.
> > > > > 
> > > > Why? oom_badness() is called after oom_scan_process_thread() returned OOM_SCAN_OK.
> > > > oom_scan_process_thread() returns OOM_SCAN_ABORT if a thread has TIF_MEMDIE set.
> > > > 
> > > 
> > > oom_scan_process_thread() checks for TIF_MEMDIE on p, not on p's threads.
> > > If one of p's threads has TIF_MEMDIE set and p does not, we actually want 
> > > to set TIF_MEMDIE for p.  That's the current behavior since it will lead 
> > > to p->mm memory freeing.  Your patch is excluding such processes entirely 
> > > and selecting another process to kill unnecessarily.
> > > 
> > 
> > I think p's threads are checked by oom_scan_process_thread() for TIF_MEMDIE
> > even if p does not have TIF_MEMDIE. What am I misunderstanding about what
> > for_each_process_thread(g, p) is doing?
> > 
> >   #define for_each_process_thread(p, t) for_each_process(p) for_each_thread(p, t)
> > 
> >   select_bad_process() {
> >     for_each_process_thread(g, p) {
> >       oom_scan_process_thread(oc, p, totalpages));
> >       oom_badness(p);
> >     }
> >   }
> > 
> 
> Yes, select_bad_process() iterates over threads, that is obvious.  The 
> point is that today it can select a thread independent of whether any of 
> its other threads have TIF_MEMDIE set, which is the desired behavior per 
> the above.  With your change, that is no longer possible because we 
> disregard _all_ threads if one of them has TIF_MEMDIE set.
> 

I still cannot understand. Today select_bad_process() can select a thread
independent of whether any of its other threads have TIF_MEMDIE set. But
select_bad_process() after all ignores that thread selected by oom_badness()
logic and aborts the iterate loop as soon as oom_scan_process_thread() finds
a TIF_MEMDIE thread from all threads. Changing oom_badness() logic to skip
processes with TIF_MEMDIE threads does not change the task select_bad_process()
logic will finally return.