linux-kernel - Re: On guest free page hinting and OOM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6b0a3610-0e7b-08dc-8b5f-707062f87bea@redhat.com>
Date:   Tue, 2 Apr 2019 19:08:57 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Alexander Duyck <alexander.duyck@...il.com>
Cc:     "Michael S. Tsirkin" <mst@...hat.com>,
        Nitesh Narayan Lal <nitesh@...hat.com>,
        kvm list <kvm@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        Paolo Bonzini <pbonzini@...hat.com>, lcapitulino@...hat.com,
        pagupta@...hat.com, wei.w.wang@...el.com,
        Yang Zhang <yang.zhang.wz@...il.com>,
        Rik van Riel <riel@...riel.com>, dodgen@...gle.com,
        Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
        dhildenb@...hat.com, Andrea Arcangeli <aarcange@...hat.com>
Subject: Re: On guest free page hinting and OOM

On 02.04.19 18:18, Alexander Duyck wrote:
> n Tue, Apr 2, 2019 at 8:57 AM David Hildenbrand <david@...hat.com> wrote:
>>
>> On 02.04.19 17:25, Michael S. Tsirkin wrote:
>>> On Tue, Apr 02, 2019 at 08:04:00AM -0700, Alexander Duyck wrote:
>>>> Basically what we would be doing is providing a means for
>>>> incrementally transitioning the buddy memory into the idle/offline
>>>> state to reduce guest memory overhead. It would require one function
>>>> that would walk the free page lists and pluck out pages that don't
>>>> have the "Offline" page type set,
>>>
>>> I think we will need an interface that gets
>>> an offline page and returns the next online free page.
>>>
>>> If we restart the list walk each time we can't guarantee progress.
>>
>> Yes, and essentially we are scanning all the time for chunks vs. we get
>> notified which chunks are possible hinting candidates. Totally different
>> design.
> 
> The problem as I see it is that we can miss notifications if we become
> too backlogged, and that will lead to us having to fall back to
> scanning anyway. So instead of trying to implement both why don't we
> just focus on the scanning approach. Otherwise the only other option
> is to hold up the guest and make it wait until the hint processing has
> completed and at that point we are back to what is essentially just a
> synchronous solution with batching anyway.
> 

In general I am not a fan of "there might be a problem, let's try
something completely different". Expect the unexpected. At this point, I
prefer to think about easy solutions to eventual problems. not
completely new designs. As I said, we've been there already.

Related to "falling behind" with hinting. If this is indeed possible
(and I'd like to know under which conditions), I wonder at which point
we no longer care about missed hints. If our guest as a lot of MM
activity, could be that is good that we are dropping hints, because our
guest is so busy, it will reuse pages soon again.

One important point is - I think - that free page hinting does not have
to fit all possible setups. In certain environments it just makes sense
to disable it. Or live with it not giving you "all the hints". E.g.
databases that eat up all free memory either way. The other extreme
would be a simple webserver that is mostly idle.

We are losing hitning of quite free memory already due to the MAX_ORDER
- X discussion. Dropping a couple of other hints shouldn't really hurt.
The question is, are there scenarios where we can completely screw up.

-- 

Thanks,

David / dhildenb