linux-kernel - Re: "mm: use early_pfn_to_nid in page_ext

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170714091304.GC2618@dhcp22.suse.cz>
Date:   Fri, 14 Jul 2017 11:13:04 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Vlastimil Babka <vbabka@...e.cz>
Cc:     Joonsoo Kim <js1304@...il.com>, Yang Shi <yang.shi@...aro.org>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: "mm: use early_pfn_to_nid in page_ext_init" broken on some
 configurations?

On Fri 07-07-17 14:00:03, Vlastimil Babka wrote:
> On 07/04/2017 07:17 AM, Joonsoo Kim wrote:
> >> 
> >> Still, backporting b8f1a75d61d8 fixes this:
> >> 
> >> [    1.538379] allocated 738197504 bytes of page_ext
> >> [    1.539340] Node 0, zone      DMA: page owner found early allocated 0 pages
> >> [    1.540179] Node 0, zone    DMA32: page owner found early allocated 33 pages
> >> [    1.611173] Node 0, zone   Normal: page owner found early allocated 96755 pages
> >> [    1.683167] Node 1, zone   Normal: page owner found early allocated 96575 pages
> >> 
> >> No panic, notice how it allocated more for page_ext, and found smaller number of
> >> early allocated pages.
> >> 
> >> Now backporting fe53ca54270a on top:
> >> 
> >> [    0.000000] allocated 738197504 bytes of page_ext
> >> [    0.000000] Node 0, zone      DMA: page owner found early allocated 0 pages
> >> [    0.000000] Node 0, zone    DMA32: page owner found early allocated 33 pages
> >> [    0.000000] Node 0, zone   Normal: page owner found early allocated 2842622 pages
> >> [    0.000000] Node 1, zone   Normal: page owner found early allocated 3694362 pages
> >> 
> >> Again no panic, and same amount of page_ext usage. But the "early allocated" numbers
> >> seem bogus to me. I think it's because init_pages_in_zone() is running and inspecting
> >> struct pages that have not been yet initialized. It doesn't end up crashing, but
> >> still doesn't seem correct?
> > 
> > Numbers looks sane to me. fe53ca54270a makes init_pages_in_zone()
> > called before page_alloc_init_late(). So, there would be many
> > uninitialized pages with PageReserved(). Page owner regarded these
> > PageReserved() page as allocated page.
> 
> That seems incorrect for two reasons:
> - init_pages_in_zone() actually skips PageReserved() pages
> - the pages don't have PageReserved() flag, until the deferred struct page init
> thread processes them via deferred_init_memmap() -> __init_single_page() AFAICS
> 
> Now I've found out why upstream reports much less early allocated pages than our
> kernel. We're missing 9d43f5aec950 ("mm/page_owner: add zone range overlapping
> check") which adds a "page_zone(page) != zone" check. I think this only works
> because the pages are not initialized and thus have no nid/zone links. Probably
> page_zone() only doesn't break because it's all zeroed. I don't think it's safe
> to rely on this?

Yes, if anything PageReserved should be checked before the zone check.

-- 
Michal Hocko
SUSE Labs