linux-kernel - Re: Thoughts on simple scanner approach for free page hinting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8032d00d-60f3-6d51-7c08-9f0f4b3e9636@redhat.com>
Date:   Mon, 8 Apr 2019 20:58:31 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Alexander Duyck <alexander.duyck@...il.com>,
        "Michael S. Tsirkin" <mst@...hat.com>
Cc:     Nitesh Narayan Lal <nitesh@...hat.com>,
        kvm list <kvm@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        Paolo Bonzini <pbonzini@...hat.com>, lcapitulino@...hat.com,
        pagupta@...hat.com, wei.w.wang@...el.com,
        Yang Zhang <yang.zhang.wz@...il.com>,
        Rik van Riel <riel@...riel.com>, dodgen@...gle.com,
        Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
        dhildenb@...hat.com, Andrea Arcangeli <aarcange@...hat.com>
Subject: Re: Thoughts on simple scanner approach for free page hinting

>>> - Define and use counters corresponding to each zone to monitor the
>>> amount of memory freed.
>>> - As soon as the 64MB free memory threshold is hit wake up the kernel
>>> thread which will scan this xbitmap and try to isolate the pages and
>>> clear the corresponding bits. (We still have to acquire zone lock to
>>> protect the respective xbitmap)
>>
>> So that's 32 pages then? I'd say just keep them in an array,
>> list, tree or hash, bitmap is for when you have nots of pages.
> 
> The xbitmap I think is for the free page tracking. The problem is this
> could build up to tons of pages while we are waiting on hints to
> complete if we have a thread that is dumping a large amount of free
> pages.
> 
>>> - Report the isolated pages back to the host in a synchronous manner.
>>> I still have to work on several details of this idea including xbitmap,
>>> but first would like to hear any suggestions/thoughts.
> 
> I'm still not a fan of trying to keep track of the free page metadata
> in real-time. It is going to be far more expensive to have every free
> and alloc have to update the extra piece of data than to just come
> through after the fact and scan the new pages that have been added.

Tracking metadata separately is a very good starting point. While
integration into core mm in some form e.g. like you describe would be
desirable long term, not messing too much with core-mm is the lower
hanging fruit.

-- 

Thanks,

David / dhildenb