linux-kernel - Re: [PATCH v10 0/6] mm / virtio: Provide support for unused page reporting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAKgT0UedoNBk3cp64SpCzXJqjtqBWZQSB7QzF7R_jhTDXbzNPg@mail.gmail.com>
Date:   Tue, 24 Sep 2019 10:07:11 -0700
From:   Alexander Duyck <alexander.duyck@...il.com>
To:     David Hildenbrand <david@...hat.com>
Cc:     Michal Hocko <mhocko@...nel.org>, virtio-dev@...ts.oasis-open.org,
        kvm list <kvm@...r.kernel.org>,
        "Michael S. Tsirkin" <mst@...hat.com>,
        Dave Hansen <dave.hansen@...el.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Matthew Wilcox <willy@...radead.org>,
        linux-mm <linux-mm@...ck.org>, Vlastimil Babka <vbabka@...e.cz>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...hsingularity.net>,
        linux-arm-kernel@...ts.infradead.org,
        Oscar Salvador <osalvador@...e.de>,
        Yang Zhang <yang.zhang.wz@...il.com>,
        Pankaj Gupta <pagupta@...hat.com>,
        Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
        Nitesh Narayan Lal <nitesh@...hat.com>,
        Rik van Riel <riel@...riel.com>, lcapitulino@...hat.com,
        "Wang, Wei W" <wei.w.wang@...el.com>,
        Andrea Arcangeli <aarcange@...hat.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Alexander Duyck <alexander.h.duyck@...ux.intel.com>
Subject: Re: [PATCH v10 0/6] mm / virtio: Provide support for unused page reporting

On Tue, Sep 24, 2019 at 8:32 AM David Hildenbrand <david@...hat.com> wrote:
>
> On 24.09.19 16:23, Michal Hocko wrote:
> > On Wed 18-09-19 10:52:25, Alexander Duyck wrote:
> > [...]
> >> In order to try and keep the time needed to find a non-reported page to
> >> a minimum we maintain a "reported_boundary" pointer. This pointer is used
> >> by the get_unreported_pages iterator to determine at what point it should
> >> resume searching for non-reported pages. In order to guarantee pages do
> >> not get past the scan I have modified add_to_free_list_tail so that it
> >> will not insert pages behind the reported_boundary.
> >>
> >> If another process needs to perform a massive manipulation of the free
> >> list, such as compaction, it can either reset a given individual boundary
> >> which will push the boundary back to the list_head, or it can clear the
> >> bit indicating the zone is actively processing which will result in the
> >> reporting process resetting all of the boundaries for a given zone.
> >
> > Is this any different from the previous version? The last review
> > feedback (both from me and Mel) was that we are not happy to have an
> > externally imposed constrains on how the page allocator is supposed to
> > maintain its free lists.
> >
> > If this is really the only way to go forward then I would like to hear
> > very convincing arguments about other approaches not being feasible.
>
> Adding to what Alexander said, I don't consider the other approaches
> (especially the bitmap-based approach Nitesh is currently working on)
> infeasible. There might be more rough edges (e.g., sparse zones) and
> eventually sometimes a little more work to be done, but definitely
> feasible. Incorporating stuff into the buddy might make some tasks
> (e.g., identify free pages) more efficient.
>
> I still somewhat like the idea of capturing hints of free pages (in
> whatever data structure) and then going over the hints, seeing if the
> pages are still free. Then only temporarily isolating the still-free
> pages, reporting them, and un-isolating them after they were reported. I
> like the idea that the pages are not fake-allocated but only temporarily
> blocked. That works nicely e.g., with the movable zone (contain only
> movable data).

One other change in this patch set is that I split the headers so that
there is an internal header that resides in the mm tree and an
external one that provides the page reporting device structure and the
register/unregister functions. All that virtio-balloon knows is that
it is registering a notifier and will be called with scatter gather
lists for memory that is not currently in use by the kernel. It has no
visibility into the internal free_areas or the current state of the
buddy allocator. Rather than having two blocks that are both trying to
maintain that state, I have consolidated it all into the buddy
allocator with page reporting.

> But anyhow, after decades of people working on free page
> hinting/reporting, I am happy with anything that gets accepted upstream :D

Agreed. After working on this for 9 months I would be happy to get
something upstream that addresses this.

- Alex