linux-kernel - Re: [External] Re: [PATCH v14 0/8] Free some vmemmap pages of HugeTLB page

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAMZfGtUDBjKqcBF3NzMp5DMx_wQKYCR0QGZ7rWUoX3DOEpXT-A@mail.gmail.com>
Date:   Fri, 5 Feb 2021 17:30:58 +0800
From:   Muchun Song <songmuchun@...edance.com>
To:     Oscar Salvador <osalvador@...e.de>
Cc:     Jonathan Corbet <corbet@....net>,
        Mike Kravetz <mike.kravetz@...cle.com>,
        Thomas Gleixner <tglx@...utronix.de>, mingo@...hat.com,
        bp@...en8.de, x86@...nel.org, hpa@...or.com,
        dave.hansen@...ux.intel.com, luto@...nel.org,
        Peter Zijlstra <peterz@...radead.org>, viro@...iv.linux.org.uk,
        Andrew Morton <akpm@...ux-foundation.org>, paulmck@...nel.org,
        mchehab+huawei@...nel.org, pawan.kumar.gupta@...ux.intel.com,
        Randy Dunlap <rdunlap@...radead.org>, oneukum@...e.com,
        anshuman.khandual@....com, jroedel@...e.de,
        Mina Almasry <almasrymina@...gle.com>,
        David Rientjes <rientjes@...gle.com>,
        Matthew Wilcox <willy@...radead.org>,
        Michal Hocko <mhocko@...e.com>,
        "Song Bao Hua (Barry Song)" <song.bao.hua@...ilicon.com>,
        David Hildenbrand <david@...hat.com>,
        HORIGUCHI NAOYA(堀口 直也) 
        <naoya.horiguchi@....com>,
        Xiongchun duan <duanxiongchun@...edance.com>,
        linux-doc@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>,
        Linux Memory Management List <linux-mm@...ck.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>
Subject: Re: [External] Re: [PATCH v14 0/8] Free some vmemmap pages of HugeTLB page

On Fri, Feb 5, 2021 at 4:59 PM Oscar Salvador <osalvador@...e.de> wrote:
>
> On Thu, Feb 04, 2021 at 11:50:35AM +0800, Muchun Song wrote:
> > Changelog in v13 -> v14:
> >   - Refuse to free the HugeTLB page when the system is under memory pressure.
> >   - Use GFP_ATOMIC to allocate vmemmap pages instead of GFP_KERNEL.
> >   - Rebase to linux-next 20210202.
> >   - Fix and add some comments for vmemmap_remap_free().
>
> What has happened to [1] ? AFAIK we still need it, right?
> If not, could you explain why?
>
> [1] https://patchwork.kernel.org/project/linux-mm/patch/20210117151053.24600-7-songmuchun@bytedance.com/

Hi Oscar,

I reply to you in another thread (in the patch #4).

Thanks. :-)

>
> >
> >   Thanks to Oscar, Mike, David H and David R's suggestions and review.
> >
> > Changelog in v12 -> v13:
> >   - Remove VM_WARN_ON_PAGE macro.
> >   - Add more comments in vmemmap_pte_range() and vmemmap_remap_free().
> >
> >   Thanks to Oscar and Mike's suggestions and review.
> >
> > Changelog in v11 -> v12:
> >   - Move VM_WARN_ON_PAGE to a separate patch.
> >   - Call __free_hugepage() with hugetlb_lock (See patch #5.) to serialize
> >     with dissolve_free_huge_page(). It is to prepare for patch #9.
> >   - Introduce PageHugeInflight. See patch #9.
> >
> > Changelog in v10 -> v11:
> >   - Fix compiler error when !CONFIG_HUGETLB_PAGE_FREE_VMEMMAP.
> >   - Rework some comments and commit changes.
> >   - Rework vmemmap_remap_free() to 3 parameters.
> >
> >   Thanks to Oscar and Mike's suggestions and review.
> >
> > Changelog in v9 -> v10:
> >   - Fix a bug in patch #11. Thanks to Oscar for pointing that out.
> >   - Rework some commit log or comments. Thanks Mike and Oscar for the suggestions.
> >   - Drop VMEMMAP_TAIL_PAGE_REUSE in the patch #3.
> >
> >   Thank you very much Mike and Oscar for reviewing the code.
> >
> > Changelog in v8 -> v9:
> >   - Rework some code. Very thanks to Oscar.
> >   - Put all the non-hugetlb vmemmap functions under sparsemem-vmemmap.c.
> >
> > Changelog in v7 -> v8:
> >   - Adjust the order of patches.
> >
> >   Very thanks to David and Oscar. Your suggestions are very valuable.
> >
> > Changelog in v6 -> v7:
> >   - Rebase to linux-next 20201130
> >   - Do not use basepage mapping for vmemmap when this feature is disabled.
> >   - Rework some patchs.
> >     [PATCH v6 08/16] mm/hugetlb: Free the vmemmap pages associated with each hugetlb page
> >     [PATCH v6 10/16] mm/hugetlb: Allocate the vmemmap pages associated with each hugetlb page
> >
> >   Thanks to Oscar and Barry.
> >
> > Changelog in v5 -> v6:
> >   - Disable PMD/huge page mapping of vmemmap if this feature was enabled.
> >   - Simplify the first version code.
> >
> > Changelog in v4 -> v5:
> >   - Rework somme comments and code in the [PATCH v4 04/21] and [PATCH v4 05/21].
> >
> >   Thanks to Mike and Oscar's suggestions.
> >
> > Changelog in v3 -> v4:
> >   - Move all the vmemmap functions to hugetlb_vmemmap.c.
> >   - Make the CONFIG_HUGETLB_PAGE_FREE_VMEMMAP default to y, if we want to
> >     disable this feature, we should disable it by a boot/kernel command line.
> >   - Remove vmemmap_pgtable_{init, deposit, withdraw}() helper functions.
> >   - Initialize page table lock for vmemmap through core_initcall mechanism.
> >
> >   Thanks for Mike and Oscar's suggestions.
> >
> > Changelog in v2 -> v3:
> >   - Rename some helps function name. Thanks Mike.
> >   - Rework some code. Thanks Mike and Oscar.
> >   - Remap the tail vmemmap page with PAGE_KERNEL_RO instead of PAGE_KERNEL.
> >     Thanks Matthew.
> >   - Add some overhead analysis in the cover letter.
> >   - Use vmemap pmd table lock instead of a hugetlb specific global lock.
> >
> > Changelog in v1 -> v2:
> >   - Fix do not call dissolve_compound_page in alloc_huge_page_vmemmap().
> >   - Fix some typo and code style problems.
> >   - Remove unused handle_vmemmap_fault().
> >   - Merge some commits to one commit suggested by Mike.
> >
> > Muchun Song (8):
> >   mm: memory_hotplug: factor out bootmem core functions to
> >     bootmem_info.c
> >   mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP
> >   mm: hugetlb: free the vmemmap pages associated with each HugeTLB page
> >   mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page
> >   mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap
> >   mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate
> >   mm: hugetlb: gather discrete indexes of tail page
> >   mm: hugetlb: optimize the code with the help of the compiler
> >
> >  Documentation/admin-guide/kernel-parameters.txt |  14 ++
> >  Documentation/admin-guide/mm/hugetlbpage.rst    |   3 +
> >  arch/x86/mm/init_64.c                           |  13 +-
> >  fs/Kconfig                                      |   6 +
> >  include/linux/bootmem_info.h                    |  65 +++++
> >  include/linux/hugetlb.h                         |  43 +++-
> >  include/linux/hugetlb_cgroup.h                  |  19 +-
> >  include/linux/memory_hotplug.h                  |  27 --
> >  include/linux/mm.h                              |   5 +
> >  mm/Makefile                                     |   2 +
> >  mm/bootmem_info.c                               | 124 ++++++++++
> >  mm/hugetlb.c                                    |  23 +-
> >  mm/hugetlb_vmemmap.c                            | 314 ++++++++++++++++++++++++
> >  mm/hugetlb_vmemmap.h                            |  33 +++
> >  mm/memory_hotplug.c                             | 116 ---------
> >  mm/sparse-vmemmap.c                             | 280 +++++++++++++++++++++
> >  mm/sparse.c                                     |   1 +
> >  17 files changed, 930 insertions(+), 158 deletions(-)
> >  create mode 100644 include/linux/bootmem_info.h
> >  create mode 100644 mm/bootmem_info.c
> >  create mode 100644 mm/hugetlb_vmemmap.c
> >  create mode 100644 mm/hugetlb_vmemmap.h
> >
> > --
> > 2.11.0
> >
> >
>
> --
> Oscar Salvador
> SUSE L3