linux-kernel - Re: (2) [PATCH] page_alloc: avoid the negative free for meminfo available

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 3 Jan 2023 11:20:03 +0100
From:   Michal Hocko <mhocko@...e.com>
To:     김재원 <jaewon31.kim@...sung.com>
Cc:     "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
        "hannes@...xchg.org" <hannes@...xchg.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "jaewon31.kim@...il.com" <jaewon31.kim@...il.com>
Subject: Re: (2) [PATCH] page_alloc: avoid the negative free for meminfo
 available

On Tue 03-01-23 18:22:32, 김재원 wrote:
> >> >On Tue 03-01-23 16:28:07, Jaewon Kim wrote:
> >> >> The totalreserve_pages could be higher than the free because of
> >> >> watermark high or watermark boost. Handle this situation and fix it to 0
> >> >> free size.
> >> >
> >> >What is the actual problem you are trying to address by this change?
> >> 
> >> Hello
> >> 
> >> As described on the original commit,
> >>   34e431b0ae39 /proc/meminfo: provide estimated available memory
> >> mm is tring to provide the avaiable memory to user space.
> >> 
> >> But if free is negative, the available memory shown to userspace
> >> would be shown smaller thatn the actual available size. The userspace
> >> may do unwanted memory shrinking actions like process kills.
> >
> >Do you have any specific example? Have you seen this happening in
> >practice or is this based on the code inspection?
> 
> I found this from a device using v5.10 based kernel.
> Actually the log was printed by user space in its format after reading /proc/meminfo.
> 
> MemFree          38220 KB
> MemAvailable     90008 KB
> Active(file)    137116 KB
> Inactive(file)  124128 KB
> SReclaimable    100960 KB
> 
> Here's /proc/zoneinfo for wmark info.
> 
> ------ ZONEINFO (/proc/zoneinfo) ------
> Node 0, zone    DMA32
>   pages free     17059
>         min      862
>         low      9790
>         high     18718
>         spanned  524288
>         present  497920
>         managed  413348
> Node 0, zone   Normal
>   pages free     12795
>         min      1044
>         low      11855
>         high     22666
>         spanned  8388608
>         present  524288
>         managed  500548
> 
> The pagecache at this time, seems to be 174,664 KB.
>   pagecache -= min(pagecache / 2, wmark_low)
> We also need to add the reclaimable and the actual free on it to be MemAvaiable.
> 
> The MemAvailable should be bigger at leat this 174,664 KB, but it was 90,008 KB only
> because the big wmark high 165,536 seems to be used.

How have you concluded that? Are you saying that a userspace would be
behaving more sanely when considering more memory to be available?
Please see more on the semantics below.

> >Also does this patch actually fix anything? Say the system is really
> >struggling and we are under min watermark. Shouldn't that lead to
> >Available to be reported as 0 without even looking at other counters?
> >
> 
> Sorry but I did not understand,

What I meant here is that the core of the high level definition says:
"An estimate of how much memory is available for starting new
applications, without swapping." If the system is close enough to watermarks 
that NR_FREE_PAGES < reserves then it is likely that further memory
allocations will not do without reclaim and potentially swapout.

So the question really is whether just clamping the value to 0 is
actually making MemAvailable more "correct"? See my point?

The actual value is never going to be lazer cut precise. Close to
watermark behavior will vary wildly depending on the memory
reclaimability. Kswapd might easily keep up with memory demand but it
also could get stuck. MemAvailable should be considered a hint rather
than an exact value IMHO.
-- 
Michal Hocko
SUSE Labs