linux-kernel - Re: can't oom-kill zap the victim's memory?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201509202350.DDG21892.FFStOLHOQOFMVJ@I-love.SAKURA.ne.jp>
Date:	Sun, 20 Sep 2015 23:50:40 +0900
From:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
To:	oleg@...hat.com, kwalker@...hat.com, cl@...ux.com,
	torvalds@...ux-foundation.org, mhocko@...nel.org
Cc:	akpm@...ux-foundation.org, rientjes@...gle.com, hannes@...xchg.org,
	vdavydov@...allels.com, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, skozina@...hat.com
Subject: Re: can't oom-kill zap the victim's memory?

Oleg Nesterov wrote:
> On 09/17, Kyle Walker wrote:
> >
> > Currently, the oom killer will attempt to kill a process that is in
> > TASK_UNINTERRUPTIBLE state. For tasks in this state for an exceptional
> > period of time, such as processes writing to a frozen filesystem during
> > a lengthy backup operation, this can result in a deadlock condition as
> > related processes memory access will stall within the page fault
> > handler.
> 
> And there are other potential reasons for deadlock.
> 
> Stupid idea. Can't we help the memory hog to free its memory? This is
> orthogonal to other improvements we can do.

So, we are trying to release memory without waiting for arriving at
exit_mm() from do_exit(), right? If it works, it will be a simple and
small change that will be easy to backport.

The idea is that since fatal_signal_pending() tasks no longer return to
user space, we can release memory allocated for use by user space, right?

Then, I think that this approach can be applied to not only OOM-kill case
but also regular kill(pid, SIGKILL) case (i.e. kick from signal_wake_up(1)
or somewhere?). A dedicated kernel thread (not limited to OOM-kill purpose)
scans for fatal_signal_pending() tasks and release that task's memory.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/