linux-kernel - Re: [RFC PATCH] mm: align anon mmap for THP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <MWHPR06MB2896ACD09C21B2939959C8A8EE800@MWHPR06MB2896.namprd06.prod.outlook.com>
Date:   Mon, 14 Jan 2019 16:29:29 +0000
From:   "Harrosh, Boaz" <Boaz.Harrosh@...app.com>
To:     "Kirill A. Shutemov" <kirill@...temov.name>,
        Mike Kravetz <mike.kravetz@...cle.com>
CC:     "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Hugh Dickins <hughd@...gle.com>,
        Michal Hocko <mhocko@...nel.org>,
        Dan Williams <dan.j.williams@...el.com>,
        Matthew Wilcox <willy@...radead.org>,
        Toshi Kani <toshi.kani@....com>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [RFC PATCH] mm: align anon mmap for THP

 Kirill A. Shutemov <kirill@...temov.name> wrote:
> On Fri, Jan 11, 2019 at 03:28:37PM -0800, Mike Kravetz wrote:
>> Ok, I just wanted to ask the question.  I've seen application code doing
>> the 'mmap sufficiently large area' then unmap to get desired alignment
>> trick.  Was wondering if there was something we could do to help.
>
> Application may want to get aligned allocation for different reasons.
> It should be okay for userspace to ask for size + (alignment - PAGE_SIZE)
> and then round up the address to get the alignment. We basically do the
> same on kernel side.
>

This is what we do and will need to keep doing for old Kernels.
But it is a pity that those holes can not be reused for small maps, and most important
that we cannot have "mapping holes" around the mapping that catch memory
overruns

> For THP, I believe, kernel already does The Right Thing™ for most users.
> User still may want to get speific range as THP (to avoid false sharing or
> something).

I'm an OK Kernel programmer.  But I was not able to create a HugePage mapping
against /dev/shm/ in a reliable way. I think it only worked on Fedora 28/29
but not on any other distro/version. (MMAP_HUGE)

We run with our own compiled Kernel on various distros, THP is configured
in but mmap against /dev/shm/ never gives me Huge pages. Does it only
work with unanimous mmap ? (I think it is mount dependent which is not
in the application control)

Just a rant. One day I will figure this out. Meanwhile I do this ugly
user mode aligns the pointers, and try to sleep at night ...

> But still I believe userspace has all required tools to get it
> right.
>

I still wish that if I ask for an mmap size aligned on 2M that I would automatically
get a 2M pointer. I don't see how the system can benefit from having both ends
of the VMA cross Huge page boundary.

> --
> Kirill A. Shutemov

Thanks
Boaz