linux-kernel - Re: Re: Re: Re: [PATCH] mm: Add nr_free_highatomimic to fix incorrect watermatk routine

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20190904081915.GH3838@dhcp22.suse.cz>
Date:   Wed, 4 Sep 2019 10:19:15 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Park Sangwoo <sangwoo2.park@....com>
Cc:     hannes@...xchg.org, arunks@...eaurora.org, guro@...com,
        richard.weiyang@...il.com, glider@...gle.com, jannh@...gle.com,
        dan.j.williams@...el.com, akpm@...ux-foundation.org,
        alexander.h.duyck@...ux.intel.com, rppt@...ux.vnet.ibm.com,
        gregkh@...uxfoundation.org, janne.huttunen@...ia.com,
        pasha.tatashin@...een.com, vbabka@...e.cz, osalvador@...e.de,
        mgorman@...hsingularity.net, khlebnikov@...dex-team.ru,
        linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: Re: Re: Re: [PATCH] mm: Add nr_free_highatomimic to fix
 incorrect watermatk routine

On Wed 04-09-19 15:54:57, Park Sangwoo wrote:
> > On Tue 03-09-19 18:59:59, Park Sangwoo wrote:
> > > On Mon 02-09-19 13:34:54, Sangwoo� wrote:
> > >>> On Fri 30-08-19 18:25:53, Sangwoo wrote:
> > >>>> The highatomic migrate block can be increased to 1% of Total memory.
> > >>>> And, this is for only highorder ( > 0 order). So, this block size is
> > >>>> excepted during check watermark if allocation type isn't alloc_harder.
> > >>>>
> > >>>> It has problem. The usage of highatomic is already calculated at
> > >>> NR_FREE_PAGES.
> > >>>>> So, if we except total block size of highatomic, it's twice minus size of
> > >>> allocated
> > >>>>> highatomic.
> > >>>>> It's cause allocation fail although free pages enough.
> > >>>>>
> > >>>>> We checked this by random test on my target(8GB RAM).
> > >>>>>
> > >>>>>  Binder:6218_2: page allocation failure: order:0, mode:0x14200ca
> > >>> (GFP_HIGHUSER_MOVABLE), nodemask=(null)
> > >>>>>  Binder:6218_2 cpuset=background mems_allowed=0
> > >>>>
> > >>>> How come this order-0 sleepable allocation fails? The upstream kernel
> > >>>> doesn't fail those allocations unless the process context is killed by
> > >>>> the oom killer.
> > >>> 
> > >>> Most calltacks are zsmalloc, as shown below.
> > >>
> > >> What makes those allocations special so that they fail unlike any other
> > >> normal order-0 requests? Also do you see the same problem with the
> > >> current upstream kernel? Is it possible this is an Android specific
> > >> issue?
> > >
> > > There is the other case of fail order-0 fail.
> > > ----
> > > hvdcp_opti: page allocation failure: order:0, mode:0x1004000(GFP_NOWAIT|__GFP_COMP), nodemask=(null)
> > 
> > This is an atomic allocation and failing that one is not a problem
> > usually. High atomic reservations might prevent GFP_NOWAIT allocation
> > from suceeding but I do not see that as a problem. This is the primary
> > purpose of the reservation. 
> 
> Thanks, your answer helped me. However, my suggestion is not to modify the use and management of the high atomic region,
> but to calculate the exact free size of the highatomic so that fail does not occur for previously shared cases.
> 
> In __zone_water_mark_ok(...) func, if it is not atomic allocation, high atomic size is excluded.
> 
> bool __zone_watermark_ok(struct zone *z,
> ...
> {
>     ...
>     if (likely(!alloc_harder)) {
>         free_pages -= z->nr_reserved_highatomic;
>     ...
> }
> 
> However, free_page excludes the size already allocated by hiahtomic.
> If highatomic block is small(Under 4GB RAM), it could be no problem.
> But, the larger the memory size, the greater the chance of problems.
> (Becasue highatomic size can be increased up to 1% of memory)

I still do not understand. NR_FREE_PAGES should include the amount of
hhighatomic reserves, right. So reducing the free_pages for normal
allocations just makes sense. Or what do I miss?

I am sorry but I find your reasoning really hard to follow.
-- 
Michal Hocko
SUSE Labs