linux-kernel - Re: Plumbers 2018 - Performance and Scalability Microconference

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <bd6f6f8b-4880-6c20-62f5-bb6ca3b5e6f7@oracle.com>
Date:   Thu, 6 Sep 2018 14:36:38 -0700
From:   Mike Kravetz <mike.kravetz@...cle.com>
To:     "Huang, Ying" <ying.huang@...el.com>,
        Christopher Lameter <cl@...ux.com>
Cc:     Daniel Jordan <daniel.m.jordan@...cle.com>,
        linux-kernel@...r.kernel.org,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        Aaron Lu <aaron.lu@...el.com>, alex.kogan@...cle.com,
        akpm@...ux-foundation.org, boqun.feng@...il.com, brouer@...hat.com,
        dave@...olabs.net, dave.dice@...cle.com,
        Dhaval Giani <dhaval.giani@...cle.com>, ktkhai@...tuozzo.com,
        ldufour@...ux.vnet.ibm.com, Pavel.Tatashin@...rosoft.com,
        paulmck@...ux.vnet.ibm.com, shady.issa@...cle.com,
        tariqt@...lanox.com, tglx@...utronix.de, tim.c.chen@...el.com,
        vbabka@...e.cz, longman@...hat.com, yang.shi@...ux.alibaba.com,
        shy828301@...il.com, subhra.mazumdar@...cle.com,
        Steven Sistare <steven.sistare@...cle.com>, jwadams@...gle.com,
        ashwinch@...gle.com, sqazi@...gle.com,
        Shakeel Butt <shakeelb@...gle.com>, walken@...gle.com,
        rientjes@...gle.com, junaids@...gle.com,
        Neha Agarwal <nehaagarwal@...gle.com>,
        Hugh Dickins <hughd@...gle.com>
Subject: Re: Plumbers 2018 - Performance and Scalability Microconference

On 09/05/2018 06:58 PM, Huang, Ying wrote:
> Hi, Christopher,
> 
> Christopher Lameter <cl@...ux.com> writes:
> 
>> On Tue, 4 Sep 2018, Daniel Jordan wrote:
>>
>>>  - Promoting huge page usage:  With memory sizes becoming ever larger, huge
>>> pages are becoming more and more important to reduce TLB misses and the
>>> overhead of memory management itself--that is, to make the system scalable
>>> with the memory size.  But there are still some remaining gaps that prevent
>>> huge pages from being deployed in some situations, such as huge page
>>> allocation latency and memory fragmentation.
>>
>> You forgot the major issue that huge pages in the page cache are not
>> supported and thus we have performance issues with fast NVME drives that
>> are now able to do 3Gbytes per sec that are only possible to reach with
>> directio and huge pages.
> 
> Yes.  That is an important gap for huge page.  Although we have huge
> page cache support for tmpfs, we lacks that for normal file systems.
> 
>> IMHO the huge page issue is just the reflection of a certain hardware
>> manufacturer inflicting pain for over a decade on its poor users by not
>> supporting larger base page sizes than 4k. No such workarounds needed on
>> platforms that support large sizes. Things just zoom along without
>> contortions necessary to deal with huge pages etc.
>>
>> Can we come up with a 2M base page VM or something? We have possible
>> memory sizes of a couple TB now. That should give us a million or so 2M
>> pages to work with.
> 
> That sounds a good idea.  Don't know whether someone has tried this.

IIRC, Hugh Dickins and some others at Google tried going down this path.
There was a brief discussion at LSF/MM.  It is something I too would like
to explore in my spare time.

-- 
Mike Kravetz