linux-kernel - Re: [RESEND PATCH 1/2 -mm] mm: account lazy free pages separately

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <79c90f6b-fcac-02e1-015a-0eaa4eafdf7d@linux.alibaba.com>
Date:   Fri, 9 Aug 2019 11:26:13 -0700
From:   Yang Shi <yang.shi@...ux.alibaba.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     kirill.shutemov@...ux.intel.com, hannes@...xchg.org,
        vbabka@...e.cz, rientjes@...gle.com, akpm@...ux-foundation.org,
        linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [RESEND PATCH 1/2 -mm] mm: account lazy free pages separately



On 8/9/19 11:02 AM, Michal Hocko wrote:
> On Fri 09-08-19 09:19:13, Yang Shi wrote:
>>
>> On 8/9/19 1:32 AM, Michal Hocko wrote:
>>> On Fri 09-08-19 07:57:44, Yang Shi wrote:
>>>> When doing partial unmap to THP, the pages in the affected range would
>>>> be considered to be reclaimable when memory pressure comes in.  And,
>>>> such pages would be put on deferred split queue and get minus from the
>>>> memory statistics (i.e. /proc/meminfo).
>>>>
>>>> For example, when doing THP split test, /proc/meminfo would show:
>>>>
>>>> Before put on lazy free list:
>>>> MemTotal:       45288336 kB
>>>> MemFree:        43281376 kB
>>>> MemAvailable:   43254048 kB
>>>> ...
>>>> Active(anon):    1096296 kB
>>>> Inactive(anon):     8372 kB
>>>> ...
>>>> AnonPages:       1096264 kB
>>>> ...
>>>> AnonHugePages:   1056768 kB
>>>>
>>>> After put on lazy free list:
>>>> MemTotal:       45288336 kB
>>>> MemFree:        43282612 kB
>>>> MemAvailable:   43255284 kB
>>>> ...
>>>> Active(anon):    1094228 kB
>>>> Inactive(anon):     8372 kB
>>>> ...
>>>> AnonPages:         49668 kB
>>>> ...
>>>> AnonHugePages:     10240 kB
>>>>
>>>> The THPs confusingly look disappeared although they are still on LRU if
>>>> you are not familair the tricks done by kernel.
>>> Is this a fallout of the recent deferred freeing work?
>> This series follows up the discussion happened when reviewing "Make deferred
>> split shrinker memcg aware".
> OK, so it is a pre-existing problem. Thanks!
>
>> David Rientjes suggested deferred split THP should be accounted into
>> available memory since they would be shrunk when memory pressure comes in,
>> just like MADV_FREE pages. For the discussion, please refer to:
>> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg2010115.html
> Thanks for the reference.
>
>>>> Accounted the lazy free pages to NR_LAZYFREE, and show them in meminfo
>>>> and other places.  With the change the /proc/meminfo would look like:
>>>> Before put on lazy free list:
>>> The name is really confusing because I have thought of MADV_FREE immediately.
>> Yes, I agree. We may use a more specific name, i.e. DeferredSplitTHP.
>>
>>>> +LazyFreePages: Cleanly freeable pages under memory pressure (i.e. deferred
>>>> +               split THP).
>>> What does that mean actually? I have hard time imagine what cleanly
>>> freeable pages mean.
>> Like deferred split THP and MADV_FREE pages, they could be reclaimed during
>> memory pressure.
>>
>> If you just go with "DeferredSplitTHP", these ambiguity would go away.
> I have to study the code some more but is there any reason why those
> pages are not accounted as proper THPs anymore? Sure they are partially
> unmaped but they are still THPs so why cannot we keep them accounted
> like that. Having a new counter to reflect that sounds like papering
> over the problem to me. But as I've said I might be missing something
> important here.

I think we could keep those pages accounted for NR_ANON_THPS since they 
are still THP although they are unmapped as you mentioned if we just 
want to fix the improper accounting.

Here the new counter is introduced for patch 2/2 to account deferred 
split THPs into available memory since NR_ANON_THPS may contain 
non-deferred split THPs.

I could use an internal counter for deferred split THPs, but if it is 
accounted by mod_node_page_state, why not just show it in /proc/meminfo? 
Or we fix NR_ANON_THPS and show deferred split THPs in /proc/meminfo?

>