Message-ID: <1437420078.3214.185.camel@hp.com>
Date:	Mon, 20 Jul 2015 13:21:18 -0600
From:	Toshi Kani <toshi.kani@...com>
To:	Ashutosh Dixit <ashutosh.dixit@...el.com>
Cc:	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"Dutt, Sudeep" <sudeep.dutt@...el.com>,
	"Rao, Nikhil" <nikhil.rao@...el.com>,
	"Williams, Dan J" <dan.j.williams@...el.com>
Subject: Re: Regression in v4.2-rc1: vmalloc_to_page with ioremap

On Mon, 2015-07-20 at 11:33 -0700, Ashutosh Dixit wrote:
> On Mon, Jul 20 2015 at 08:59:02 AM, Toshi Kani <toshi.kani@...com> wrote:
> 
> > Can you also try the 'nohugeiomap' kernel option to the 4.2-rc1 kernel
> > to see if the problem goes away?
> 
> Yes the problem goes away with 'nohugeiomap'.
> 
> > Yes, vmalloc_to_page() assumes 4KB mappings.  But ioremap with huge
> > pages was enabled in v4.1, while v4.2-rc1 has update for the check with
> > MTRRs.
> 
> Yes we had bisected the failure down to the following commit, so the
> problem goes away if this commit is reverted. So we think that without 
> this
> commit ioremap actually maps device memory in 4K pages and that is why we
> don't see the issue.
> 
> commit b73522e0c1be58d3c69b124985b8ccf94e3677f7
> Author: Toshi Kani <toshi.kani@...com>
> Date:   Tue May 26 10:28:10 2015 +0200
> 
>     x86/mm/mtrr: Enhance MTRR checks in kernel mapping helpers

Thanks for checking into this!  That makes sense.  Your system has the MTRR
default type set to UC, which covers device memory ranges.  In v4.1, large
page mappings are not allowed for the ranges covered by MTRRs with non-WB
types.  In v4.2, large page mappings are allowed as long as the request
type does not conflict with MTRRs.  So, v4.2 enabled large page mappings to
the driver.

> > Can you send me outputs of the following files?  If the driver
> > fails to load in v4.2-rc1, you can obtain the info in v4.1.
> > 
> >   /proc/mtrr
> >   /proc/iomem
> >   /proc/vmallocinfo
> >   /sys/kernel/debug/kernel_page_tables (need CONFIG_X86_PTDUMP set)
> 
> Since the outputs are large I have sent you the outputs in a separate 
> mail
> outside the mailing list.

Did you collect the info with your driver loaded?  I did not see ioremap
requests from your driver in the vmallocinfo output.

> > Also, does the driver map a regular memory range with ioremap?  If not,
> > how does 'struct page' get allocated for the range (since
> > vmalloc_to_page returns a page pointer)?
> 
> No the driver does not map regular memory with ioremap, only device
> memory. vmalloc_to_page was returning a valid 'struct page' in this case
> too. It appears it can do this correctly using pte_page as long as all
>  four
> page table levels (pgd, pud, pmd, pte) are present and the problem seems 
> to
> be happening because in the case of huge pages they are not. For us the 
> bar
> size is 8 G so we think that with the new ioremap maps the bars using 1 G
> pages.

Well, it is probably not a valid `struct page` pointer you got from
vmalloc_to_page...  pte_page(pte) simply adds a pfn to vmemmap.  So, yes,
you get a pointer, which can be put back to the pfn by subtracting vmemmap.
 It may look fine, but this pointer does not point to a struct page unless
it is part of the regular memory (or you somehow allocated struct page for
the range).  Can you check to see if the struct page pointer from
vmalloc_to_page actually points a struct page entry?

  /* memmap is virtually contiguous.  */
  #define __pfn_to_page(pfn)      (vmemmap + (pfn))
  #define __page_to_pfn(page)     (unsigned long)((page) - vmemmap)

Thanks,
-Toshi

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists