linux-kernel - Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.10.1707111336250.60183@chino.kir.corp.google.com>
Date:   Tue, 11 Jul 2017 13:40:04 -0700 (PDT)
From:   David Rientjes <rientjes@...gle.com>
To:     Michal Hocko <mhocko@...nel.org>
cc:     linux-mm@...ck.org,
        Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>,
        Oleg Nesterov <oleg@...hat.com>,
        Andrea Argangeli <andrea@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [RFC PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Tue, 11 Jul 2017, Michal Hocko wrote:

> This?
> ---
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 5dc0ff22d567..e155d1d8064f 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -470,11 +470,14 @@ static bool __oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm)
>  {
>  	struct mmu_gather tlb;
>  	struct vm_area_struct *vma;
> -	bool ret = true;
>  
>  	if (!down_read_trylock(&mm->mmap_sem))
>  		return false;
>  
> +	/* There is nothing to reap so bail out without signs in the log */
> +	if (!mm->mmap)
> +		goto unlock;
> +
>  	/*
>  	 * Tell all users of get_user/copy_from_user etc... that the content
>  	 * is no longer stable. No barriers really needed because unmapping
> @@ -508,9 +511,10 @@ static bool __oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm)
>  			K(get_mm_counter(mm, MM_ANONPAGES)),
>  			K(get_mm_counter(mm, MM_FILEPAGES)),
>  			K(get_mm_counter(mm, MM_SHMEMPAGES)));
> +unlock:
>  	up_read(&mm->mmap_sem);
>  
> -	return ret;
> +	return true;
>  }
>  
>  #define MAX_OOM_REAP_RETRIES 10

Yes, this folded in with the original RFC patch appears to work better 
with light testing.

However, I think MAX_OOM_REAP_RETRIES and/or the timeout of HZ/10 needs to 
be increased as well to address the issue that Tetsuo pointed out.  The 
oom reaper shouldn't be required to do any work unless it is resolving a 
livelock, and that scenario should be relatively rare.  The oom killer 
being a natural ultra slow path, I think it would be justifiable to wait 
longer or retry more times than simply 1 second before declaring that 
reaping is not possible.  It reduces the likelihood of additional oom 
killing.