linux-kernel - Re: [RFC PATCH v2 0/5] hugetlb: parallelize hugetlb page init on boot

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <55c6c1f6-0792-61c3-86ed-4729d4a3fdf5@google.com>
Date:   Tue, 12 Dec 2023 16:10:22 -0800 (PST)
From:   David Rientjes <rientjes@...gle.com>
To:     Mike Kravetz <mike.kravetz@...cle.com>
cc:     Gang Li <gang.li@...ux.dev>, David Hildenbrand <david@...hat.com>,
        Muchun Song <muchun.song@...ux.dev>,
        Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, ligang.bdlg@...edance.com
Subject: Re: [RFC PATCH v2 0/5] hugetlb: parallelize hugetlb page init on
 boot

On Tue, 12 Dec 2023, Mike Kravetz wrote:

> On 12/12/23 14:14, David Rientjes wrote:
> > On Fri, 8 Dec 2023, Gang Li wrote:
> > 
> > > Hi all, hugetlb init parallelization has now been updated to v2.
> > > 
> > > To David Hildenbrand: padata multithread utilities has been used to reduce
> > > code complexity.
> > > 
> > > To David Rientjes: The patch for measuring time will be separately included
> > > in the reply. Please test during your free time, thanks.
> > > 
> > 
> > I'd love to, but what kernel is this based on?  :)  I can't get this to 
> > apply to any kernels that I have recently benchmarked with.
> 
> I was able to apply and build on top of v6.7-rc5.
> 
> Gang Li,
> Since hugetlb now depends on CONFIG_PADATA, the Kconfig file should be
> updated to reflect this.

Gotcha, thanks.

I got this:

ld: error: undefined symbol: padata_do_multithreaded
referenced by hugetlb.c:3470 (./mm/hugetlb.c:3470)
              vmlinux.o:(gather_bootmem_prealloc)
referenced by hugetlb.c:3592 (./mm/hugetlb.c:3592)
              vmlinux.o:(hugetlb_hstate_alloc_pages_non_gigantic)
referenced by hugetlb.c:3599 (./mm/hugetlb.c:3599)
              vmlinux.o:(hugetlb_hstate_alloc_pages_non_gigantic)

So, yeah we need to enable DEFERRED_STRUCT_PAGE_INIT for this to build.

On 6.6 I measured "hugepagesz=1G hugepages=11776" on as 12TB host to be 
77s this time around.

A latest Linus build with this patch set does not boot successfully, so 
I'll need to look into that and try to capture the failure.  Not sure if 
it's related to this patch or the latest Linus build in general.