linux-kernel - Re: [PATCH 03/13] mm: shmem: provide oom badness for shmem files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YqdFkfLVFUD5K6EK@dhcp22.suse.cz>
Date:   Mon, 13 Jun 2022 16:11:29 +0200
From:   Michal Hocko <mhocko@...e.com>
To:     Christian König 
        <ckoenig.leichtzumerken@...il.com>
Cc:     Christian König <christian.koenig@....com>,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        intel-gfx@...ts.freedesktop.org, amd-gfx@...ts.freedesktop.org,
        nouveau@...ts.freedesktop.org, linux-tegra@...r.kernel.org,
        linux-fsdevel@...r.kernel.org, linux-mm@...ck.org,
        alexander.deucher@....com, daniel@...ll.ch,
        viro@...iv.linux.org.uk, akpm@...ux-foundation.org,
        hughd@...gle.com, andrey.grodzovsky@....com
Subject: Re: [PATCH 03/13] mm: shmem: provide oom badness for shmem files

On Mon 13-06-22 14:55:54, Christian König wrote:
> Am 13.06.22 um 14:11 schrieb Michal Hocko:
> > [SNIP]
> > > > > Alternative I could try to track the "owner" of a buffer (e.g. a shmem
> > > > > file), but then it can happen that one processes creates the object and
> > > > > another one is writing to it and actually allocating the memory.
> > > > If you can enforce that the owner is really responsible for the
> > > > allocation then all should be fine. That would require MAP_POPULATE like
> > > > semantic and I suspect this is not really feasible with the existing
> > > > userspace. It would be certainly hard to enforce for bad players.
> > > I've tried this today and the result was: "BUG: Bad rss-counter state
> > > mm:000000008751d9ff type:MM_FILEPAGES val:-571286".
> > > 
> > > The problem is once more that files are not informed when the process
> > > clones. So what happened is that somebody called fork() with an mm_struct
> > > I've accounted my pages to. The result is just that we messed up the
> > > rss_stats and  the the "BUG..." above.
> > > 
> > > The key difference between normal allocated pages and the resources here is
> > > just that we are not bound to an mm_struct in any way.
> > It is not really clear to me what exactly you have tried.
> 
> I've tried to track the "owner" of a driver connection by keeping a
> reference to the mm_struct who created this connection inside our file
> private and then use add_mm_counter() to account all the allocations of the
> driver to this mm_struct.
> 
> This works to the extend that now the right process is killed in an OOM
> situation. The problem with this approach is that the driver is not informed
> about operations like fork() or clone(), so what happens is that after a
> fork()/clone() we have an unbalanced rss-counter.

Yes, I do not think you can make per-process accounting without a
concept of the per-process ownership.

> Let me maybe get back to the initial question: We have resources which are
> not related to the virtual address space of a process, how should we tell
> the OOM killer about them?

I would say memcg, but we have discussed this already...

I do not think that exposing a resource (in a form of a counter
or something like that) is sufficient. The existing oom killer
implementation is hevily process centric (with memcg extension for
grouping but not changing the overall design in principle). If you
want to make it aware of resources which are not directly accounted to
processes then a a new implementation is necessary IMHO. You would need
to evaluate those resources and kill all the tasks that can hold on that
resource.

This is also the reason why I am not really fan of the per file
badness because it adds a notion of resource that is not process bound
in general so it will add all sorts of weird runtime corner cases which
are impossible to anticipate [*]. Maybe that will work in some scenarios
but definitely not something to be done by default without users opting
into that and being aware of consequences. 

There have been discussions that the existing oom implementation cannot
fit all potential usecases so maybe we need to finally decide to use a
plugable, BPFable etc architecture allow implementations that fit
specific needs.

[*] I know it is not directly related but kinda similar. In the past
we used to have heuristics to consider work done as a resource . That is
kill younger processes preferably to reduce the damage.  This has turned
out to have a very unpredictable behavior and many complains by
users. Situation has improved when the selection was solely based on
rss. This has its own cons of course but at least they are predictable.
-- 
Michal Hocko
SUSE Labs