linux-kernel - Re: [PATCH] mm, oom: allow oom reaper to race with exit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 27 Jul 2017 16:55:59 +0200
From:   Andrea Arcangeli <aarcange@...hat.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     "Kirill A. Shutemov" <kirill@...temov.name>,
        Andrew Morton <akpm@...ux-foundation.org>,
        David Rientjes <rientjes@...gle.com>,
        Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
        Oleg Nesterov <oleg@...hat.com>,
        Hugh Dickins <hughd@...gle.com>, linux-mm@...ck.org,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] mm, oom: allow oom reaper to race with exit_mmap

On Thu, Jul 27, 2017 at 08:50:24AM +0200, Michal Hocko wrote:
> Yes this will work and it won't depend on the oom_lock. But isn't it
> just more ugly than simply doing
> 
> 	if (tsk_is_oom_victim) {
> 		down_write(&mm->mmap_sem);
> 		locked = true;
> 	}
> 	free_pgtables(...)
> 	[...]
> 	if (locked)
> 		down_up(&mm->mmap_sem);

To me not doing if (tsk_is_oom...) { down_write; up_write } is by
default a confusing implementation, because it's not strict and not
strict code is not self documenting and you've to think twice of why
you're doing something the way you're doing it.

The doubt on what was the point to hold the mmap_sem during
free_pgtables is precisely why I started digging into this issue
because it didn't look possible you could truly benefit from holding
the mmap_sem during free_pgtables.

I also don't like having a new invariant that your solution relies on,
that is mm->mmap = NULL, when we can make just set the MMF_OOM_SKIP a
bit earlier that it gets set anyway and use that to control the other
side of the race.

I like strict code that uses as fewer invariants as possible and that
never holds a lock for any instruction more than it is required (again
purely for self documenting reasons, the CPU won't notice much one
instruction more or less).

Even with your patch the two branches are unnecessary, that may not be
measurable, but it's still wasted CPU. It's all about setting mm->mmap
before the up_write. In fact my patch should at least put an incremental
unlikely around my single branch added to exit_mmap.

I see the {down_write;up_write} Hugh's ksm_exit-like as a strict
solution to this issue and I wrote it specifically while trying to
research a way to be more strict because from the start it didn't look
the holding of the mmap_sem during free_pgtables was necessary.

I'm also fine to drop the oom_lock but I think it can be done
incrementally as it's a separate issue, my second patch should allow
for it with no adverse side effects.

All I care about is the exit_mmap path because it runs too many times
not to pay deep attention to every bit of it ;).

Thanks,
Andrea