Message-ID: <CA+CK2bA_Cb95mu6XRLy3nnjNoRwMHR2OQuRAPGU-uzPXbBth1g@mail.gmail.com>
Date:   Wed, 30 Oct 2019 13:53:52 -0400
From:   Pavel Tatashin <pasha.tatashin@...een.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     Vincent Whitchurch <vincent.whitchurch@...s.com>,
        "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
        "osalvador@...e.de" <osalvador@...e.de>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] mm/sparse: Consistently do not zero memmap

On Wed, Oct 30, 2019 at 1:31 PM Michal Hocko <mhocko@...nel.org> wrote:
>
> On Wed 30-10-19 12:53:41, Pavel Tatashin wrote:
> > On Wed, Oct 30, 2019 at 11:31 AM Michal Hocko <mhocko@...nel.org> wrote:
> > >
> > > On Wed 30-10-19 11:20:44, Pavel Tatashin wrote:
> > > > On Wed, Oct 30, 2019 at 10:13 AM Michal Hocko <mhocko@...nel.org> wrote:
> > > > >
> > > > > [Add Pavel - the email thread starts http://lkml.kernel.org/r/20191030131122.8256-1-vincent.whitchurch@axis.com
> > > > >  but it used your old email address]
> > > > >
> > > > > On Wed 30-10-19 15:02:16, Vincent Whitchurch wrote:
> > > > > > On Wed, Oct 30, 2019 at 02:29:58PM +0100, Michal Hocko wrote:
> > > > > > > On Wed 30-10-19 14:11:22, Vincent Whitchurch wrote:
> > > > > > > > (I noticed this because on my ARM64 platform, with 1 GiB of memory the
> > > > > > > >  first [and only] section is allocated from the zeroing path while with
> > > > > > > >  2 GiB of memory the first 1 GiB section is allocated from the
> > > > > > > >  non-zeroing path.)
> > > > > > >
> > > > > > > Do I get it right that sparse_buffer_init couldn't allocate memmap for
> > > > > > > the full node for some reason and so sparse_init_nid would have to
> > > > > > > allocate one for each memory section?
> > > > > >
> > > > > > Not quite.  The sparsemap_buf is successfully allocated with the correct
> > > > > > size in sparse_buffer_init(), but sparse_buffer_alloc() fails to
> > > > > > allocate the same size from it.
> > > > > >
> > > > > > The reason it fails is that sparse_buffer_alloc() for some reason wants
> > > > > > to return a pointer which is aligned to the allocation size.  But the
> > > > > > sparsemap_buf was only allocated with PAGE_SIZE alignment so there's not
> > > > > > enough space to align it.
> > > > > >
> > > > > > I don't understand the reason for this alignment requirement since the
> > > > > > fallback path also allocates with PAGE_SIZE alignment.  I'm guessing the
> > > > > > alignment is for the VMEMAP code which also uses sparse_buffer_alloc()?
> > > > >
> > > > > I am not 100% sure TBH. Aligning makes some sense when mapping the
> > > > > memmaps to page tables but that would suggest that sparse_buffer_init
> > > > > is using a wrong alignment then. It is quite wasteful to allocate
> > > > > alarge misaligned block like that.
> > > > >
> > > > > Your patch still makes sense but this is something to look into.
> > > > >
> > > > > Pavel?
> > > >
> > > > I remember thinking about this large alignment, as it looked out of
> > > > place to me also.
> > > > It was there to keep memmap in single chunks on larger x86 machines.
> > > > Perhaps it can be revisited now.
> > >
> > > Don't we need 2MB aligned memmaps for their PMD mappings?
> >
> > Yes, PMD_SIZE should be the alignment here. It just does not make
> > sense to align to size.
>
> What about this? It still aligns to the size but that should be
> correctly done to the section size level.
>
> diff --git a/mm/sparse.c b/mm/sparse.c
> index 72f010d9bff5..ab1e6175ac9a 100644
> --- a/mm/sparse.c
> +++ b/mm/sparse.c
> @@ -456,8 +456,7 @@ struct page __init *__populate_section_memmap(unsigned long pfn,
>         if (map)
>                 return map;
>
> -       map = memblock_alloc_try_nid(size,
> -                                         PAGE_SIZE, addr,
> +       map = memblock_alloc_try_nid(size, size, addr,
>                                           MEMBLOCK_ALLOC_ACCESSIBLE, nid);
>         if (!map)
>                 panic("%s: Failed to allocate %lu bytes align=0x%lx nid=%d from=%pa\n",
> @@ -474,8 +473,13 @@ static void __init sparse_buffer_init(unsigned long size, int nid)
>  {
>         phys_addr_t addr = __pa(MAX_DMA_ADDRESS);
>         WARN_ON(sparsemap_buf); /* forgot to call sparse_buffer_fini()? */
> +       /*
> +        * Pre-allocated buffer is mainly used by __populate_section_memmap
> +        * and we want it to be properly aligned to the section size - this is
> +        * especially the case for VMEMMAP which maps memmap to PMDs
> +        */
>         sparsemap_buf =
> -               memblock_alloc_try_nid_raw(size, PAGE_SIZE,
> +               memblock_alloc_try_nid_raw(size, section_map_size(),
>                                                 addr,
>                                                 MEMBLOCK_ALLOC_ACCESSIBLE, nid);
>         sparsemap_buf_end = sparsemap_buf + size;

This looks good, I think we should also change alignment in fallback
of vmemmap_alloc_block() to be
section_map_size().

+++ b/mm/sparse-vmemmap.c
@@ -65,9 +65,10 @@ void * __meminit vmemmap_alloc_block(unsigned long
size, int node)
                        warned = true;
                }
                return NULL;
-       } else
-               return __earlyonly_bootmem_alloc(node, size, size,
+       } else {
+               return __earlyonly_bootmem_alloc(node, size, section_map_size(),
                                __pa(MAX_DMA_ADDRESS));
+       }
 }

Pasha


>
> --
> Michal Hocko
> SUSE Labs

Powered by blists - more mailing lists