linux-kernel - Re: [v3] fs/proc/task_mmu: Implement IOCTL for efficient page table scanning

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CABb0KFEr_CDZyvZ27q2b7DbXwW3h+hNLjjzBw1GzzkZW=j-Dow@mail.gmail.com>
Date:   Thu, 27 Jul 2023 13:18:19 +0200
From:   Michał Mirosław <emmir@...gle.com>
To:     Paul Gofman <pgofman@...eweavers.com>
Cc:     Muhammad Usama Anjum <usama.anjum@...labora.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Michał Mirosław <mirq-linux@...e.qmqm.pl>,
        Andrei Vagin <avagin@...il.com>,
        Danylo Mocherniuk <mdanylo@...gle.com>,
        Alex Sierra <alex.sierra@....com>,
        Alexander Viro <viro@...iv.linux.org.uk>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Axel Rasmussen <axelrasmussen@...gle.com>,
        Christian Brauner <brauner@...nel.org>,
        Cyrill Gorcunov <gorcunov@...il.com>,
        Dan Williams <dan.j.williams@...el.com>,
        David Hildenbrand <david@...hat.com>,
        Greg KH <gregkh@...uxfoundation.org>,
        "Gustavo A . R . Silva" <gustavoars@...nel.org>,
        "Liam R . Howlett" <Liam.Howlett@...cle.com>,
        Matthew Wilcox <willy@...radead.org>,
        Mike Rapoport <rppt@...nel.org>, Nadav Amit <namit@...are.com>,
        Pasha Tatashin <pasha.tatashin@...een.com>,
        Peter Xu <peterx@...hat.com>, Shuah Khan <shuah@...nel.org>,
        Suren Baghdasaryan <surenb@...gle.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Yang Shi <shy828301@...il.com>,
        Yun Zhou <yun.zhou@...driver.com>,
        linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        linux-mm@...ck.org, linux-kselftest@...r.kernel.org,
        kernel@...labora.com
Subject: Re: [v3] fs/proc/task_mmu: Implement IOCTL for efficient page table scanning

On Thu, 27 Jul 2023 at 01:06, Paul Gofman <pgofman@...eweavers.com> wrote:
>
> Hello Michał,
>
>      I was looking into that from the Wine point of view and did a bit
> of testing, so will try to answer the question cited below.

Thanks for the extensive explanation!

>      Without Windows large pages I guess the only way to make this work
> correctly is to disable THP with madvise(MADV_NOHUGEPAGE) on the memory
> ranges allocated with MEM_WRITE_WATCH, as the memory changes should not
> only be reported but also tracked with 4k page granularity as Windows
> applications expect.
>
>      Currently we don't implement MEM_LARGE_PAGES flag support in Wine
> (while of course might want to do that in the future). On Windows using
> this flag requires special permissions and implies more than just using
> huge pages under the hood but also, in particular, locking pages in
> memory. I'd expect that support to be extended in Windows though in the
> future in some way. WRT write watches, the range is watched with large
> page granularity. GetWriteWatch lpdwGranularity output parameter returns
> the value of "large page minimum" (returned by GetLargePageMinimum) and
> the returned addresses correspond to those large pages. I suppose to
> implement that on top of Linux huge pages we'd need a way to control
> huge pages allocation at the first place, i. e., a way to enforce the
> specified size for the huge pages for the memory ranged being mapped.
> Without that I am afraid the only way to correctly implement that is to
> still disable THP on the range and only adjust our API output so that
> matches expected.
>
>      Not related to the question, but without any relation to Wine and
> Windows API current way of dealing with THP in the API design looks a
> bit not straightforward to me. In a sense that transparent huge pages
> will appear not so transparent when it comes to dirty pages tracking. If
> I understand correctly, the application which allocated a reasonably big
> memory area and didn't use madvise(MADV_NOHUGEPAGE) might end up with a
> whole range being a single page and getting dirtified as a whole, which
> may likely void app's optimization based on changed memory tracking. Not
> that I know an ideal way out of this, maybe it is a matter of having THP
> disabled by default on watched ranges or clearly warning about this
> caveat in documentation?

So, this means that the max_pages limit should count HugeTLB pages as
1 not HPAGE_SIZE/PAGE_SIZE pages.
Also, to get this right, we might need another PAGE_IS_HUGETLB
category, so that userspace can differentiate the ranges if needed.

Is it possible (on Windows) to have MEM_LARGE_PAGES allocation near a
normal one and run GetWriteWatch() on both in one call? If so, how
does it behave / what is expected?

Best Regards
Michał Mirosław