linux-kernel - Re: [RFC PATCH 00/16] 1GB THP support on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200903170157.GE14765@casper.infradead.org>
Date:   Thu, 3 Sep 2020 18:01:57 +0100
From:   Matthew Wilcox <willy@...radead.org>
To:     Jason Gunthorpe <jgg@...dia.com>
Cc:     Roman Gushchin <guro@...com>, Michal Hocko <mhocko@...e.com>,
        Zi Yan <ziy@...dia.com>, linux-mm@...ck.org,
        Rik van Riel <riel@...riel.com>,
        "Kirill A . Shutemov" <kirill.shutemov@...ux.intel.com>,
        Shakeel Butt <shakeelb@...gle.com>,
        Yang Shi <yang.shi@...ux.alibaba.com>,
        David Nellans <dnellans@...dia.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH 00/16] 1GB THP support on x86_64

On Thu, Sep 03, 2020 at 01:50:51PM -0300, Jason Gunthorpe wrote:
> At least from a RDMA NIC perspective I've heard from a lot of users
> that higher order pages at the DMA level is giving big speed ups too.
> 
> It is basically the same dynamic as CPU TLB, except missing a 'TLB'
> cache in a PCI-E device is dramatically more expensive to refill. With
> 200G and soon 400G networking these misses are a growing problem.
> 
> With HPC nodes now pushing 1TB of actual physical RAM and single
> applications basically using all of it, there is definately some
> meaningful return - if pages can be reliably available.
> 
> At least for HPC where the node returns to an idle state after each
> job and most of the 1TB memory becomes freed up again, it seems more
> believable to me that a large cache of 1G pages could be available?

You may be interested in trying out my current THP patchset:

http://git.infradead.org/users/willy/pagecache.git

It doesn't allocate pages larger than PMD size, but it does allocate pages
*up to* PMD size for the page cache which means that larger pages are
easier to create as larger pages aren't fragmented all over the system.

If someone wants to opportunistically allocate pages larger than PMD
size, I've put some preliminary support in for that, but I've never
tested any of it.  That's not my goal at the moment.

I'm not clear whether these HPC users primarily use page cache or
anonymous memory (with O_DIRECT).  Probably a mixture.