linux-kernel - Re: [External] Re: [PATCH v2 00/19] Free some vmemmap pages of hugetlb page

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20201030151926.GL1478@dhcp22.suse.cz>
Date:   Fri, 30 Oct 2020 16:19:26 +0100
From:   Michal Hocko <mhocko@...e.com>
To:     Muchun Song <songmuchun@...edance.com>
Cc:     Jonathan Corbet <corbet@....net>,
        Mike Kravetz <mike.kravetz@...cle.com>,
        Thomas Gleixner <tglx@...utronix.de>, mingo@...hat.com,
        bp@...en8.de, x86@...nel.org, hpa@...or.com,
        dave.hansen@...ux.intel.com, luto@...nel.org,
        Peter Zijlstra <peterz@...radead.org>, viro@...iv.linux.org.uk,
        Andrew Morton <akpm@...ux-foundation.org>, paulmck@...nel.org,
        mchehab+huawei@...nel.org, pawan.kumar.gupta@...ux.intel.com,
        Randy Dunlap <rdunlap@...radead.org>, oneukum@...e.com,
        anshuman.khandual@....com, jroedel@...e.de,
        Mina Almasry <almasrymina@...gle.com>,
        David Rientjes <rientjes@...gle.com>,
        Matthew Wilcox <willy@...radead.org>,
        Xiongchun duan <duanxiongchun@...edance.com>,
        linux-doc@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>,
        Linux Memory Management List <linux-mm@...ck.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>
Subject: Re: [External] Re: [PATCH v2 00/19] Free some vmemmap pages of
 hugetlb page

On Fri 30-10-20 18:24:25, Muchun Song wrote:
> On Fri, Oct 30, 2020 at 5:14 PM Michal Hocko <mhocko@...e.com> wrote:
> >
> > On Mon 26-10-20 22:50:55, Muchun Song wrote:
> > > If we uses the 1G hugetlbpage, we can save 4095 pages. This is a very
> > > substantial gain. On our server, run some SPDK/QEMU applications which
> > > will use 1000GB hugetlbpage. With this feature enabled, we can save
> > > ~16GB(1G hugepage)/~11GB(2MB hugepage) memory.
> > [...]
> > >  15 files changed, 1091 insertions(+), 165 deletions(-)
> > >  create mode 100644 include/linux/bootmem_info.h
> > >  create mode 100644 mm/bootmem_info.c
> >
> > This is a neat idea but the code footprint is really non trivial. To a
> > very tricky code which hugetlb is unfortunately.
> >
> > Saving 1,6% of memory is definitely interesting especially for 1GB pages
> > which tend to be more static and where the savings are more visible.
> >
> > Anyway, I haven't seen any runtime overhead analysis here. What is the
> > price to modify the vmemmap page tables and make them pte rather than
> > pmd based (especially for 2MB hugetlb). Also, how expensive is the
> > vmemmap page tables reconstruction on the freeing path?
> 
> Yeah, I haven't tested the remapping overhead of reserving a hugetlb
> page. I can do that. But the overhead is not on the allocation/freeing of
> each hugetlb page, it is only once when we reserve some hugetlb pages
> through /proc/sys/vm/nr_hugepages. Once the reservation is successful,
> the subsequent allocation, freeing and using are the same as before
> (not patched).

Yes, that is quite clear. Except for the hugetlb overcommit and
migration if the pool is depeleted. Maybe few other cases.

> So I think that the overhead is acceptable.

Having some numbers for a such a large feature is really needed.
-- 
Michal Hocko
SUSE Labs