linux-kernel - Re: [PATCH/RFC] mm: do not regard CMA pages as free on watermark check

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:	Thu, 10 Sep 2015 08:39:24 +0200
From:	Vlastimil Babka <vbabka@...e.cz>
To:	Vitaly Wool <vitalywool@...il.com>,
	Laura Abbott <labbott@...hat.com>
Cc:	Vitaly Wool <vwool@...mail.com>,
	LKML <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	Joonsoo Kim <iamjoonsoo.kim@....com>,
	Mel Gorman <mgorman@...hsingularity.net>
Subject: Re: [PATCH/RFC] mm: do not regard CMA pages as free on watermark
 check

[CC Joonsoo, Mel]

On 09/09/2015 08:31 PM, Vitaly Wool wrote:
> Hi Laura,
> 
> On Wed, Sep 9, 2015 at 7:56 PM, Laura Abbott <labbott@...hat.com> wrote:
> 
>> (cc-ing linux-mm)
>> On 09/09/2015 07:44 AM, Vitaly Wool wrote:
>>
>>> __zone_watermark_ok() does not corrrectly take high-order
>>> CMA pageblocks into account: high-order CMA blocks are not
>>> removed from the watermark check. Moreover, CMA pageblocks
>>> may suddenly vanish through CMA allocation, so let's not
>>> regard these pages as free in __zone_watermark_ok().
>>>
>>> This patch also adds some primitive testing for the method
>>> implemented which has proven that it works as it should.
>>>
>>>
>> The choice to include CMA as part of watermarks was pretty deliberate.
>> Do you have a description of the problem you are facing with
>> the watermark code as is? Any performance numbers?
>>
>>
> let's start with facing the fact that the calculation in
> __zone_watermark_ok() is done incorrectly for the case when ALLOC_CMA is
> not set. While going through pages by order it is implicitly considered

You're not the first who tried to fix it, I think Joonsoo tried as well?
I think the main objection was against further polluting fastpaths due to CMA.

Note that Mel has a patchset removing high-order watermark checks (in the last
patch of https://lwn.net/Articles/655406/ ) so this will be moot afterwards.

> that CMA pages can be used and this impacts the result of the function.
> 
> This can be solved in a slightly different way compared to what I proposed
> but it needs per-order CMA pages accounting anyway. Then it would have
> looked like:
> 
>         for (o = 0; o < order; o++) {
>                 /* At the next order, this order's pages become unavailable
> */
>                 free_pages -= z->free_area[o].nr_free << o;
> #ifdef CONFIG_CMA
>                 if (!(alloc_flags & ALLOC_CMA))
>                         free_pages -= z->free_area[o].nr_free_cma << o;
>                 /* Require fewer higher order pages to be free */
>                 min >>= 1;
> ...
> 
> But what we have also seen is that CMA pages may suddenly disappear due to
> CMA allocator work so the whole watermark checking was still unreliable,
> causing compaction to not run when it ought to and thus leading to

Well, watermark checking is inherently racy. CMA pages disappearing is no
exception, non-CMA pages may disappear as well.

> (otherwise redundant) low memory killer operations, so I decided to propose
> a safer method instead.
> 
> Best regards,
>    Vitaly
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/