linux-kernel - Re: [PATCH v1] mm: Add /proc/$PID/pageflags

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Tue, 2 Nov 2021 11:38:23 -0700
From:   Mina Almasry <almasrymina@...gle.com>
To:     David Hildenbrand <david@...hat.com>
Cc:     Nathan Lewis <npl@...gle.com>, Yu Zhao <yuzhao@...gle.com>,
        "Paul E . McKenney" <paulmckrcu@...com>,
        Jonathan Corbet <corbet@....net>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Peter Xu <peterx@...hat.com>,
        Ivan Teterevkov <ivan.teterevkov@...anix.com>,
        Florian Schmidt <florian.schmidt@...anix.com>,
        linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        linux-mm@...ck.org
Subject: Re: [PATCH v1] mm: Add /proc/$PID/pageflags

On Tue, Nov 2, 2021 at 4:42 AM David Hildenbrand <david@...hat.com> wrote:
>
> >> Bit 58-60 are still free, no? Bit 57 was recently added for uffd-wp
> >> purposes I think.
> >>
> >> #define PM_SOFT_DIRTY           BIT_ULL(55)
> >> #define PM_MMAP_EXCLUSIVE       BIT_ULL(56)
> >> #define PM_UFFD_WP              BIT_ULL(57)
> >> #define PM_FILE                 BIT_ULL(61)
> >> #define PM_SWAP                 BIT_ULL(62)
> >> #define PM_PRESENT              BIT_ULL(63)
> >>
> >> PM_MMAP_EXCLUSIVE and PM_FILE already go into the direction of "what is
> >> mapped" IMHO. So just a thought if something in there (PM_HUGE? PM_THP?)
> >> ... could make sense.
> >>
> >
> > Thanks! I _think_ that would work for us, I'll look into confirming.
> > To be honest I still wonder if eventually different folks will find
> > uses for other page flags and eventually we'll run out of pagemaps
> > bits, but I'll yield to whatever you think is best here.
>
> Using one of the remaining 3 bits should be fine. In the worst case,
> we'll need pagemap_ext at some point that provides more bits per PFN, if
> we ever run out of bits.
>

That sounds great to me. Thank you Both Matthew and David for
patiently explaining the concerns with /proc/self/pageflags to me and
suggesting alternatives that could work :-)

> But as mentioned by Matthew, extending mincore() could also work: not
> only indicating if the page is resident, but also in which "form" it is
> resident.
>

I need to learn more about mincore() to be honest, from casually
reading some docs I didn't get a full understanding on if/why that
would work better. I'll do some investigating and upload V2 either
with /proc/self/pagemaps or mincore() and why I chose such and we can
go from there.

> We could separate the cases "cont PTE huge page" vs. "PMD huge page".
>

So to be completely honest (and I need to confirm), we are using this
on x86 and we essentially care that the virt address is mapped by 2MB,
so mapped by PMD. I think (but need to confirm) that's what the
pageflags HUGE bit refers to as well as does PageHuge() and
TransPageHuge(). After confirming I'll upload V2 with the precise info
we need (I think it's going to be "PMD huge page" as David says).

> I recall that the information (THP / !THP) might be valuable for users:
> there was a discussion to let user space decide where to place THP.
> (IIRC madvise() extension to have something like MADV_COLLAPSE_THP /
> MADV_DISSOLVE_THP)
>
> --
> Thanks,
>
> David / dhildenb
>