linux-kernel - Re: [patch v2] mm, thp: always specify ineligible vmas as nh in smaps

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 25 Sep 2018 12:52:09 -0700 (PDT)
From:   David Rientjes <rientjes@...gle.com>
To:     Vlastimil Babka <vbabka@...e.cz>
cc:     Michal Hocko <mhocko@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Alexey Dobriyan <adobriyan@...il.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        linux-api@...r.kernel.org
Subject: Re: [patch v2] mm, thp: always specify ineligible vmas as nh in
 smaps

On Mon, 24 Sep 2018, Vlastimil Babka wrote:

> On 9/24/18 10:02 PM, Michal Hocko wrote:
> > On Mon 24-09-18 21:56:03, Michal Hocko wrote:
> >> On Mon 24-09-18 12:30:07, David Rientjes wrote:
> >>> Commit 1860033237d4 ("mm: make PR_SET_THP_DISABLE immediately active")
> >>> introduced a regression in that userspace cannot always determine the set
> >>> of vmas where thp is ineligible.
> >>>
> >>> Userspace relies on the "nh" flag being emitted as part of /proc/pid/smaps
> >>> to determine if a vma is eligible to be backed by hugepages.
> >>
> >> I was under impression that nh resp hg flags only tell about the madvise
> >> status. How do you exactly use these flags in an application?
> >>

This is used to identify heap mappings that should be able to fault thp 
but do not, and they normally point to a low-on-memory or fragmentation 
issue.  After commit 1860033237d4, our users of PR_SET_THP_DISABLE no 
longer show "nh" for their heap mappings so they get reported as having a 
low thp ratio when in reality it is disabled.  It is also used in 
automated testing to ensure that vmas get disabled for thp appropriately 
and we used "nh" since that is how PR_SET_THP_DISABLE previously enforced 
this, and those tests now break.

> >> Your eligible rules as defined here:
> >>
> >>> + [*] A process mapping is eligible to be backed by transparent hugepages (thp)
> >>> +     depending on system-wide settings and the mapping itself.  See
> >>> +     Documentation/admin-guide/mm/transhuge.rst for default behavior.  If a
> >>> +     mapping has a flag of "nh", it is not eligible to be backed by hugepages
> >>> +     in any condition, either because of prctl(PR_SET_THP_DISABLE) or
> >>> +     madvise(MADV_NOHUGEPAGE).  PR_SET_THP_DISABLE takes precedence over any
> >>> +     MADV_HUGEPAGE.
> >>
> >> doesn't seem to match the reality. I do not see all the file backed
> >> mappings to be nh marked. So is this really about eligibility rather
> >> than the madvise status? Maybe it is just the above documentation that
> >> needs to be updated.
> 
> Yeah the change from madvise to eligibility in the doc seems to go too far.
> 

I'll reword this to explicitly state that "hg" and "nh" mappings either 
allow or disallow thp backing.