linux-kernel - RE: (2) [PATCH] page_alloc: avoid the negative free for meminfo available

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20230103103950epcms1p5fc40605dd0f3165f86d7fbaff78ff87b@epcms1p5>
Date:   Tue, 03 Jan 2023 19:39:50 +0900
From:   김재원 <jaewon31.kim@...sung.com>
To:     Michal Hocko <mhocko@...e.com>
CC:     "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
        "hannes@...xchg.org" <hannes@...xchg.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "jaewon31.kim@...il.com" <jaewon31.kim@...il.com>
Subject: RE: (2) [PATCH] page_alloc: avoid the negative free for meminfo
 available

>> >> >On Tue 03-01-23 16:28:07, Jaewon Kim wrote:
>> >> >> The totalreserve_pages could be higher than the free because of
>> >> >> watermark high or watermark boost. Handle this situation and fix it to 0
>> >> >> free size.
>> >> >
>> >> >What is the actual problem you are trying to address by this change?
>> >> 
>> >> Hello
>> >> 
>> >> As described on the original commit,
>> >>   34e431b0ae39 /proc/meminfo: provide estimated available memory
>> >> mm is tring to provide the avaiable memory to user space.
>> >> 
>> >> But if free is negative, the available memory shown to userspace
>> >> would be shown smaller thatn the actual available size. The userspace
>> >> may do unwanted memory shrinking actions like process kills.
>> >
>> >Do you have any specific example? Have you seen this happening in
>> >practice or is this based on the code inspection?
>> 
>> I found this from a device using v5.10 based kernel.
>> Actually the log was printed by user space in its format after reading /proc/meminfo.
>> 
>> MemFree          38220 KB
>> MemAvailable     90008 KB
>> Active(file)    137116 KB
>> Inactive(file)  124128 KB
>> SReclaimable    100960 KB
>> 
>> Here's /proc/zoneinfo for wmark info.
>> 
>> ------ ZONEINFO (/proc/zoneinfo) ------
>> Node 0, zone    DMA32
>>   pages free     17059
>>         min      862
>>         low      9790
>>         high     18718
>>         spanned  524288
>>         present  497920
>>         managed  413348
>> Node 0, zone   Normal
>>   pages free     12795
>>         min      1044
>>         low      11855
>>         high     22666
>>         spanned  8388608
>>         present  524288
>>         managed  500548
>> 
>> The pagecache at this time, seems to be 174,664 KB.
>>   pagecache -= min(pagecache / 2, wmark_low)
>> We also need to add the reclaimable and the actual free on it to be MemAvaiable.
>> 
>> The MemAvailable should be bigger at leat this 174,664 KB, but it was 90,008 KB only
>> because the big wmark high 165,536 seems to be used.
>
>How have you concluded that? Are you saying that a userspace would be
>behaving more sanely when considering more memory to be available?
>Please see more on the semantics below.
>
>> >Also does this patch actually fix anything? Say the system is really
>> >struggling and we are under min watermark. Shouldn't that lead to
>> >Available to be reported as 0 without even looking at other counters?
>> >
>> 
>> Sorry but I did not understand,
>
>What I meant here is that the core of the high level definition says:
>"An estimate of how much memory is available for starting new
>applications, without swapping." If the system is close enough to watermarks 
>that NR_FREE_PAGES < reserves then it is likely that further memory
>allocations will not do without reclaim and potentially swapout.

Yes reclaim would be needed in that case.

I think it is just a matter of perspective.
If I follow you, I think, the totalreserve_pages should be considered
as must-have free size.

>
>So the question really is whether just clamping the value to 0 is
>actually making MemAvailable more "correct"? See my point?
>
>The actual value is never going to be lazer cut precise. Close to
>watermark behavior will vary wildly depending on the memory
>reclaimability. Kswapd might easily keep up with memory demand but it
>also could get stuck. MemAvailable should be considered a hint rather
>than an exact value IMHO.

Yeah correct, it is not perfect.
I will drop my patch.
It was nice discussion.
Thank you