linux-kernel - Re: Possible deny of service with memfd

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 4 Feb 2021 18:12:56 +0100
From:   Michal Hocko <mhocko@...e.com>
To:     Christian König <christian.koenig@....com>
Cc:     LKML <linux-kernel@...r.kernel.org>
Subject: Re: Possible deny of service with memfd_create()

On Thu 04-02-21 17:32:20, Christian König wrote:
> Hi Michal,
> 
> as requested in the other mail thread the following sample code gets my test
> system down within seconds.
> 
> The issue is that the memory allocated for the file descriptor is not
> accounted to the process allocating it, so the OOM killer pics whatever
> process it things is good but never my small test program.
> 
> Since memfd_create() doesn't need any special permission this is a rather
> nice deny of service and as far as I can see also works with a standard
> Ubuntu 5.4.0-65-generic kernel.

Thanks for following up. This is really nasty but now that I am looking
at it more closely, this is not really different from tmpfs in general.
You are free to create files and eat the memory without being accounted
for that memory because that is not seen as your memory from the sysstem
POV. You would have to map that memory to be part of your rss.

The only existing protection right now is to use memoery cgroup
controller because the tmpfs memory is accounted to the process which
faults the memory in (or write to the file).

I am not sure there is a good way to handle this in general
unfortunatelly. Shmem is is just tricky (e.g. how to you deal with left
overs after the fd is closed?). Maybe memfd_create can be more clever
and account memory to all owners of the fd but even that sounds far from
trivial from the accounting POV. It is true that tmpfs can at least
control who can write to it which is not the case for memfd but then we
hit the backward compatibility wall.
-- 
Michal Hocko
SUSE Labs