Message-ID: <20210222215502.GB1741768@linux.ibm.com>
Date:   Mon, 22 Feb 2021 23:55:02 +0200
From:   Mike Rapoport <rppt@...ux.ibm.com>
To:     George Kennedy <george.kennedy@...cle.com>
Cc:     David Hildenbrand <david@...hat.com>,
        Andrey Konovalov <andreyknvl@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Catalin Marinas <catalin.marinas@....com>,
        Vincenzo Frascino <vincenzo.frascino@....com>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Konrad Rzeszutek Wilk <konrad@...nok.org>,
        Will Deacon <will.deacon@....com>,
        Andrey Ryabinin <aryabinin@...tuozzo.com>,
        Alexander Potapenko <glider@...gle.com>,
        Marco Elver <elver@...gle.com>,
        Peter Collingbourne <pcc@...gle.com>,
        Evgenii Stepanov <eugenis@...gle.com>,
        Branislav Rankov <Branislav.Rankov@....com>,
        Kevin Brodsky <kevin.brodsky@....com>,
        Christoph Hellwig <hch@...radead.org>,
        kasan-dev <kasan-dev@...glegroups.com>,
        Linux ARM <linux-arm-kernel@...ts.infradead.org>,
        Linux Memory Management List <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Dhaval Giani <dhaval.giani@...cle.com>
Subject: Re: [PATCH] mm, kasan: don't poison boot memory

On Mon, Feb 22, 2021 at 01:42:56PM -0500, George Kennedy wrote:
> 
> On 2/22/2021 11:13 AM, David Hildenbrand wrote:
> > On 22.02.21 16:13, George Kennedy wrote:
> > > 
> > > On 2/22/2021 4:52 AM, David Hildenbrand wrote:
> > > > 
> > > > Let me look into the code ... I have little experience with ACPI
> > > > details, so bear with me.
> > > > 
> > > > I assume that acpi_map()/acpi_unmap() map some firmware blob that is
> > > > provided via firmware/bios/... to us.
> > > > 
> > > > should_use_kmap() tells us whether
> > > > a) we have a "struct page" and should kmap() that one
> > > > b) we don't have a "struct page" and should ioremap.
> > > > 
> > > > As it is a blob, the firmware should always reserve that memory region
> > > > via memblock (e.g., memblock_reserve()), such that we either
> > > > 1) don't create a memmap ("struct page") at all (-> case b) )
> > > > 2) if we have to create e memmap, we mark the page PG_reserved and
> > > >     *never* expose it to the buddy (-> case a) )
> > > > 
> > > > 
> > > > Are you telling me that in this case we might have a memmap for the HW
> > > > blob that is *not* PG_reserved? In that case it most probably got
> > > > exposed to the buddy where it can happily get allocated/freed.
> > > > 
> > > > The latent BUG would be that that blob gets exposed to the system like
> > > > ordinary RAM, and not reserved via memblock early during boot.
> > > > Assuming that blob has a low physical address, with my patch it will
> > > > get allocated/used a lot earlier - which would mean we trigger this
> > > > latent BUG now more easily.
> > > > 
> > > > There have been similar latent BUGs on ARM boards that my patch
> > > > discovered where special RAM regions did not get marked as reserved
> > > > via the device tree properly.
> > > > 
> > > > Now, this is just a wild guess :) Can you dump the page when mapping
> > > > (before PageReserved()) and when unmapping, to see what the state of
> > > > that memmap is?
> > > 
> > > Thank you David for the explanation and your help on this,
> > > 
> > > dump_page() before PageReserved and before kmap() in the above patch:
> > > 
> > > [    1.116480] ACPI: Core revision 20201113
> > > [    1.117628] XXX acpi_map: about to call kmap()...
> > > [    1.118561] page:ffffea0002f914c0 refcount:0 mapcount:0
> > > mapping:0000000000000000 index:0x0 pfn:0xbe453
> > > [    1.120381] flags: 0xfffffc0000000()
> > > [    1.121116] raw: 000fffffc0000000 ffffea0002f914c8 ffffea0002f914c8
> > > 0000000000000000
> > > [    1.122638] raw: 0000000000000000 0000000000000000 00000000ffffffff
> > > 0000000000000000
> > > [    1.124146] page dumped because: acpi_map pre SetPageReserved
> > > 
> > > I also added dump_page() before unmapping, but it is not hit. The
> > > following for the same pfn now shows up I believe as a result of setting
> > > PageReserved:
> > > 
> > > [   28.098208] BUG:Bad page state in process mo dprobe pfn:be453
> > > [   28.098394] page:ffffea0002f914c0 refcount:0 mapcount:0
> > > mapping:0000000000000000 index:0x1 pfn:0xbe453
> > > [   28.098394] flags: 0xfffffc0001000(reserved)
> > > [   28.098394] raw: 000fffffc0001000 dead000000000100 dead000000000122
> > > 0000000000000000
> > > [   28.098394] raw: 0000000000000001 0000000000000000 00000000ffffffff
> > > 0000000000000000
> > > [   28.098394] page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
> > > [   28.098394] page_owner info is not present (never set?)
> > > [   28.098394] Modules linked in:
> > > [   28.098394] CPU: 2 PID: 204 Comm: modprobe Not tainted
> > > 5.11.0-3dbd5e3 #66
> > > [   28.098394] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > > BIOS 0.0.0 02/06/2015
> > > [   28.098394] Call Trace:
> > > [   28.098394]  dump_stack+0xdb/0x120
> > > [   28.098394]  bad_page.cold.108+0xc6/0xcb
> > > [   28.098394]  check_new_page_bad+0x47/0xa0
> > > [   28.098394]  get_page_from_freelist+0x30cd/0x5730
> > > [   28.098394]  ? __isolate_free_page+0x4f0/0x4f0
> > > [   28.098394]  ? init_object+0x7e/0x90
> > > [   28.098394]  __alloc_pages_nodemask+0x2d8/0x650
> > > [   28.098394]  ? write_comp_data+0x2f/0x90
> > > [   28.098394]  ? __alloc_pages_slowpath.constprop.103+0x2110/0x2110
> > > [   28.098394]  ? __sanitizer_cov_trace_pc+0x21/0x50
> > > [   28.098394]  alloc_pages_vma+0xe2/0x560
> > > [   28.098394]  do_fault+0x194/0x12c0
> > > [   28.098394]  ? write_comp_data+0x2f/0x90
> > > [   28.098394]  __handle_mm_fault+0x1650/0x26c0
> > > [   28.098394]  ? copy_page_range+0x1350/0x1350
> > > [   28.098394]  ? write_comp_data+0x2f/0x90
> > > [   28.098394]  ? write_comp_data+0x2f/0x90
> > > [   28.098394]  handle_mm_fault+0x1f9/0x810
> > > [   28.098394]  ? write_comp_data+0x2f/0x90
> > > [   28.098394]  do_user_addr_fault+0x6f7/0xca0
> > > [   28.098394]  exc_page_fault+0xaf/0x1a0
> > > [   28.098394]  asm_exc_page_fault+0x1e/0x30
> > > [   28.098394] RIP: 0010:__clear_user+0x30/0x60
> > 
> > I think the PAGE_FLAGS_CHECK_AT_PREP check in this instance means that
> > someone is trying to allocate that page with the PG_reserved bit set.
> > This means that the page actually was exposed to the buddy.
> > 
> > However, when you SetPageReserved(), I don't think that PG_buddy is set
> > and the refcount is 0. That could indicate that the page is on the buddy
> > PCP list. Could be that it is getting reused a couple of times.
> > 
> > The PFN 0xbe453 looks a little strange, though. Do we expect ACPI tables
> > close to 3 GiB ? No idea. Could it be that you are trying to map a wrong
> > table? Just a guess.
> > 
> > > 
> > > What would be  the correct way to reserve the page so that the above
> > > would not be hit?
> > 
> > I would have assumed that if this is a binary blob, that someone (which
> > I think would be acpi code) reserved via memblock_reserve() early during
> > boot.
> > 
> > E.g., see drivers/acpi/tables.c:acpi_table_upgrade()->memblock_reserve().
> 
> acpi_table_upgrade() gets called, but bails out before memblock_reserve() is
> called. Thus, it appears no pages are getting reserved.

acpi_table_upgrade() does not actually reserve memory but rather open
codes memblock allocation with memblock_find_in_range() +
memblock_reserve(), so it does not seem related anyway.

Do you have by chance a full boot log handy? 
 
>     503 void __init acpi_table_upgrade(void)
>     504 {

...

>     568         if (table_nr == 0)
>     569                 return;                                 <-- bails
> out here
> "drivers/acpi/tables.c"
> 
> George
> 

-- 
Sincerely yours,
Mike.

Powered by blists - more mailing lists