linux-kernel - [PATCH 0/3] fix few OOM victim allocation runaways

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Wed,  1 Feb 2017 10:27:03 +0100
From:   Michal Hocko <mhocko@...nel.org>
To:     Andrew Morton <akpm@...ux-foundation.org>
Cc:     Christoph Hellwig <hch@....de>,
        Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
        Al Viro <viro@...iv.linux.org.uk>, <linux-mm@...ck.org>,
        <linux-fsdevel@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: [PATCH 0/3] fix few OOM victim allocation runaways

Hi,
these three patches tried to address a simple OOM victim runaways when
the oom victim can deplete the memory reserve completely. Tetsuo was able
to trigger the depletion in the write(2) path and I believe the similar
is possible for the read part. Vmalloc would be a bit harder but still
not impossible.

Unfortunately I do not see a better way around this issue as long as we
give OOM victims access to memory reserves without any limits. I have
tried to limit this access [1] which would help at least to keep some
memory for emergency actions. Anyway, even if we limit the amount of
reserves the OOM victim can consume it is still preferable to back off
before accessible reserves are depleted.

Tetsuo was suggesting introducing __GFP_KILLABLE which would fail the
allocation rather than consuming the reserves. I see two problems with
this approach.
        1) in order this flags work as expected all the blocking
        operations in the allocator call chain (including the direct
        reclaim) would have to be killable and this is really non
        trivial to achieve. Especially when we do not have any control
        over shrinkers.
        2) even if the above could be dealt with we would still have to
        find all the places which do allocation in the loop based on
        the user request. So it wouldn't be simpler than an explicit
        fatal_signal_pending check.

Thoughts?
Michal Hocko (3):
      fs: break out of iomap_file_buffered_write on fatal signals
      mm, fs: check for fatal signals in do_generic_file_read
      vmalloc: back of when the current is killed

 fs/dax.c     | 5 +++++
 fs/iomap.c   | 3 +++
 mm/filemap.c | 5 +++++
 mm/vmalloc.c | 5 +++++
 4 files changed, 18 insertions(+)

[1] http://lkml.kernel.org/r/20161004090009.7974-2-mhocko@kernel.org