linux-kernel - Re: [RFC] Per file OOM badness

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <e1ec9d7b-99e3-1e2a-cb47-5e9a0a383703@amd.com>
Date:   Tue, 30 Jan 2018 12:56:10 +0100
From:   Christian König <christian.koenig@....com>
To:     Michel Dänzer <michel@...nzer.net>,
        Nicolai Hähnle <nhaehnle@...il.com>,
        Michal Hocko <mhocko@...nel.org>, Roman Gushchin <guro@...com>
Cc:     linux-mm@...ck.org, dri-devel@...ts.freedesktop.org,
        linux-kernel@...r.kernel.org, amd-gfx@...ts.freedesktop.org
Subject: Re: [RFC] Per file OOM badness

Am 30.01.2018 um 12:42 schrieb Michel Dänzer:
> On 2018-01-30 12:36 PM, Nicolai Hähnle wrote:
>> On 30.01.2018 12:34, Michel Dänzer wrote:
>>> On 2018-01-30 12:28 PM, Christian König wrote:
>>>> Am 30.01.2018 um 12:02 schrieb Michel Dänzer:
>>>>> On 2018-01-30 11:40 AM, Christian König wrote:
>>>>>> Am 30.01.2018 um 10:43 schrieb Michel Dänzer:
>>>>>>> [SNIP]
>>>>>>>> Would it be ok to hang onto potentially arbitrary mmget references
>>>>>>>> essentially forever? If that's ok I think we can do your process
>>>>>>>> based
>>>>>>>> account (minus a few minor inaccuracies for shared stuff perhaps,
>>>>>>>> but no
>>>>>>>> one cares about that).
>>>>>>> Honestly, I think you and Christian are overthinking this. Let's try
>>>>>>> charging the memory to every process which shares a buffer, and go
>>>>>>> from
>>>>>>> there.
>>>>>> My problem is that this needs to be bullet prove.
>>>>>>
>>>>>> For example imagine an application which allocates a lot of BOs, then
>>>>>> calls fork() and let the parent process die. The file descriptor lives
>>>>>> on in the child process, but the memory is not accounted against the
>>>>>> child.
>>>>> What exactly are you referring to by "the file descriptor" here?
>>>> The file descriptor used to identify the connection to the driver. In
>>>> other words our drm_file structure in the kernel.
>>>>
>>>>> What happens to BO handles in general in this case? If both parent and
>>>>> child process keep the same handle for the same BO, one of them
>>>>> destroying the handle will result in the other one not being able to
>>>>> use
>>>>> it anymore either, won't it?
>>>> Correct.
>>>>
>>>> That usage is actually not useful at all, but we already had
>>>> applications which did exactly that by accident.
>>>>
>>>> Not to mention that somebody could do it on purpose.
>>> Can we just prevent child processes from using their parent's DRM file
>>> descriptors altogether? Allowing it seems like a bad idea all around.
>> Existing protocols pass DRM fds between processes though, don't they?
>>
>> Not child processes perhaps, but special-casing that seems like awful
>> design.
> Fair enough.
>
> Can we disallow passing DRM file descriptors which have any buffers
> allocated? :)

Hehe good point, but I'm sorry I have to ruin that.

The root VM page table is allocated when the DRM file descriptor is 
created and we want to account those to whoever uses the file descriptor 
as well.

We could now make an exception for the root VM page table to not be 
accounted (shouldn't be that much compared to the rest of the VM tree), 
but Nicolai is right all those exceptions are just an awful design :)

Looking into the fs layer there actually only seem to be two function 
which are involved when a file descriptor is installed/removed from a 
process. So we just need to add some callbacks there.

Regards,
Christian.