linux-kernel - Re: PROBLEM: Remapping hugepages mappings causes kernel to return EINVAL

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20171027173154.a5msarm2qzlee3od@node.shutemov.name>
Date:   Fri, 27 Oct 2017 20:31:54 +0300
From:   "Kirill A. Shutemov" <kirill@...temov.name>
To:     Vlastimil Babka <vbabka@...e.cz>
Cc:     "C.Wehrmeyer" <c.wehrmeyer@....de>,
        Michal Hocko <mhocko@...nel.org>,
        Mike Kravetz <mike.kravetz@...cle.com>, linux-mm@...ck.org,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Andrea Arcangeli <aarcange@...hat.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
Subject: Re: PROBLEM: Remapping hugepages mappings causes kernel to return
 EINVAL

On Fri, Oct 27, 2017 at 04:29:16PM +0200, Vlastimil Babka wrote:
> On 10/24/2017 09:41 AM, C.Wehrmeyer wrote:
> > On 2017-10-23 20:02, Michal Hocko wrote:
> >> On Mon 23-10-17 19:52:27, C.Wehrmeyer wrote:
> >> [...]
> >>>> or you can mmap a larger block and
> >>>> munmap the initial unaligned part.
> >>>
> >>> And how is that supposed to be transparent? When I hear "transparent" I
> >>> think of a mechanism which I can put under a system so that it benefits from
> >>> it, while the system does not notice or at least does not need to be aware
> >>> of it. The system also does not need to be changed for it.
> >>
> >> How do you expect to get a huge page when the mapping itself is not
> >> properly aligned?
> > 
> > There are four ways that I can think of from the top of my head, but 
> > only one of them would be actually transparent.
> > 
> > 1. Provide a flag to mmap, which might be something different from 
> > MAP_HUGETLB. After all your question revolved merely around properly 
> > aligned pages - we don't want to *force* the kernel to reserve 
> > hugepages, we just want it to provide the proper alignment in this case. 
> > That wouldn't be very transparent, but it would be the easiest route to 
> > go (and mmap already kind-of supports such a thing).
> 
> Maybe just have mmap() detect that the requested size is a multiple of
> huge page size, and then align it automatically? I.e. a heuristic that
> should work in 99% of the cases?

Just don't bother.

Anon mapping for appliaction that would really benefit THP would grow
naturally: kernel will allocation new mapping next to the old one and
merge them. Doing fancy things here may hurt performance due to going
number of VMAs.

And we already do right thing for file mapping (tmpfs/shmem):
->get_unmapped_area would provide the right spot for the file, given the
size of mapping and ->vm_pgoff.

-- 
 Kirill A. Shutemov