linux-kernel - Re: [PATCH] mm, oom: allow oom reaper to race with exit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.10.1708141719090.50317@chino.kir.corp.google.com>
Date:   Mon, 14 Aug 2017 17:20:58 -0700 (PDT)
From:   David Rientjes <rientjes@...gle.com>
To:     Andrea Arcangeli <aarcange@...hat.com>
cc:     Michal Hocko <mhocko@...nel.org>,
        "Kirill A. Shutemov" <kirill@...temov.name>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>,
        Oleg Nesterov <oleg@...hat.com>,
        Hugh Dickins <hughd@...gle.com>, linux-mm@...ck.org,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Wed, 26 Jul 2017, Andrea Arcangeli wrote:

> From 3d9001490ee1a71f39c7bfaf19e96821f9d3ff16 Mon Sep 17 00:00:00 2001
> From: Andrea Arcangeli <aarcange@...hat.com>
> Date: Tue, 25 Jul 2017 20:02:27 +0200
> Subject: [PATCH 1/1] mm: oom: let oom_reap_task and exit_mmap to run
>  concurrently
> 
> This is purely required because exit_aio() may block and exit_mmap() may
> never start, if the oom_reap_task cannot start running on a mm with
> mm_users == 0.
> 
> At the same time if the OOM reaper doesn't wait at all for the memory
> of the current OOM candidate to be freed by exit_mmap->unmap_vmas, it
> would generate a spurious OOM kill.
> 
> If it wasn't because of the exit_aio or similar blocking functions in
> the last mmput, it would be enough to change the oom_reap_task() in
> the case it finds mm_users == 0, to wait for a timeout or to wait for
> __mmput to set MMF_OOM_SKIP itself, but it's not just exit_mmap the
> problem here so the concurrency of exit_mmap and oom_reap_task is
> apparently warranted.
> 
> It's a non standard runtime, exit_mmap() runs without mmap_sem, and
> oom_reap_task runs with the mmap_sem for reading as usual (kind of
> MADV_DONTNEED).
> 
> The race between the two is solved with a combination of
> tsk_is_oom_victim() (serialized by task_lock) and MMF_OOM_SKIP
> (serialized by a dummy down_write/up_write cycle on the same lines of
> the ksm_exit method).
> 
> If the oom_reap_task() may be running concurrently during exit_mmap,
> exit_mmap will wait it to finish in down_write (before taking down mm
> structures that would make the oom_reap_task fail with use after
> free).
> 
> If exit_mmap comes first, oom_reap_task() will skip the mm if
> MMF_OOM_SKIP is already set and in turn all memory is already freed
> and furthermore the mm data structures may already have been taken
> down by free_pgtables.
> 
> Signed-off-by: Andrea Arcangeli <aarcange@...hat.com>

With your follow-up one liner to include linux/oom.h folded in:

Tested-by: David Rientjes <rientjes@...gle.com>