linux-kernel - Re: [PATCH 03/13] mm: shmem: provide oom badness for shmem files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 9 Jun 2022 17:22:14 +0200
From:   Christian König <christian.koenig@....com>
To:     Felix Kuehling <felix.kuehling@....com>,
        Michal Hocko <mhocko@...e.com>,
        Christian König <ckoenig.leichtzumerken@...il.com>
Cc:     linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        intel-gfx@...ts.freedesktop.org, amd-gfx@...ts.freedesktop.org,
        nouveau@...ts.freedesktop.org, linux-tegra@...r.kernel.org,
        linux-fsdevel@...r.kernel.org, linux-mm@...ck.org,
        alexander.deucher@....com, daniel@...ll.ch,
        viro@...iv.linux.org.uk, akpm@...ux-foundation.org,
        hughd@...gle.com, andrey.grodzovsky@....com
Subject: Re: [PATCH 03/13] mm: shmem: provide oom badness for shmem files

Am 09.06.22 um 17:19 schrieb Felix Kuehling:
>
> Am 2022-06-09 um 10:21 schrieb Michal Hocko:
>> On Thu 09-06-22 16:10:33, Christian König wrote:
>>> Am 09.06.22 um 14:57 schrieb Michal Hocko:
>>>> On Thu 09-06-22 14:16:56, Christian König wrote:
>>>>> Am 09.06.22 um 11:18 schrieb Michal Hocko:
>>>>>> On Tue 31-05-22 11:59:57, Christian König wrote:
>>>>>>> This gives the OOM killer an additional hint which processes are
>>>>>>> referencing shmem files with potentially no other accounting for 
>>>>>>> them.
>>>>>>>
>>>>>>> Signed-off-by: Christian König <christian.koenig@....com>
>>>>>>> ---
>>>>>>>     mm/shmem.c | 6 ++++++
>>>>>>>     1 file changed, 6 insertions(+)
>>>>>>>
>>>>>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>>>>>> index 4b2fea33158e..a4ad92a16968 100644
>>>>>>> --- a/mm/shmem.c
>>>>>>> +++ b/mm/shmem.c
>>>>>>> @@ -2179,6 +2179,11 @@ unsigned long 
>>>>>>> shmem_get_unmapped_area(struct file *file,
>>>>>>>         return inflated_addr;
>>>>>>>     }
>>>>>>> +static long shmem_oom_badness(struct file *file)
>>>>>>> +{
>>>>>>> +    return i_size_read(file_inode(file)) >> PAGE_SHIFT;
>>>>>>> +}
>>>>>> This doesn't really represent the in memory size of the file, 
>>>>>> does it?
>>>>> Well the file could be partially or fully swapped out as anonymous 
>>>>> memory or
>>>>> the address space only sparse populated, but even then just using 
>>>>> the file
>>>>> size as OOM badness sounded like the most straightforward approach 
>>>>> to me.
>>>> It covers hole as well, right?
>>> Yes, exactly.
>> So let's say I have a huge sparse shmem file. I will get killed because
>> the oom_badness of such a file would be large as well...
>
> Would killing processes free shmem files, though? Aren't those 
> persistent anyway? In that case, shmem files should not contribute to 
> oom_badness at all.

At least for the memfd_create() case they do, yes.

Those files were never part of any filesystem in the first place, so by 
killing all the process referencing them you can indeed free the memory 
locked by them.

Regards,
Christian.

>
> I guess a special case would be files that were removed from the 
> filesystem but are still open in some processes.
>
> Regards,
>   Felix
>
>
>>
>>>>> What could happen is that the file is also mmaped and we double 
>>>>> account.
>>>>>
>>>>>> Also the memcg oom handling could be considerably skewed if the 
>>>>>> file was
>>>>>> shared between more memcgs.
>>>>> Yes, and that's one of the reasons why I didn't touched the memcg 
>>>>> by this
>>>>> and only affected the classic OOM killer.
>>>> oom_badness is for all oom handlers, including memcg. Maybe I have
>>>> misread an earlier patch but I do not see anything specific to global
>>>> oom handling.
>>> As far as I can see the oom_badness() function is only used in
>>> oom_kill.c and in procfs to return the oom score. Did I missed
>>> something?
>> oom_kill.c implements most of the oom killer functionality. Memcg oom
>> killing is a part of that. Have a look at select_bad_process.
>>