linux-kernel - Re: RFC: Memory Tiering Kernel Interfaces

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <52541497-c097-5a51-4718-feed13660255@intel.com>
Date:   Wed, 4 May 2022 10:02:20 -0700
From:   Dave Hansen <dave.hansen@...el.com>
To:     Wei Xu <weixugc@...gle.com>
Cc:     Alistair Popple <apopple@...dia.com>,
        Davidlohr Bueso <dave@...olabs.net>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Huang Ying <ying.huang@...el.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Yang Shi <shy828301@...il.com>, Linux MM <linux-mm@...ck.org>,
        Greg Thelen <gthelen@...gle.com>,
        "Aneesh Kumar K.V" <aneesh.kumar@...ux.ibm.com>,
        Jagdish Gediya <jvgediya@...ux.ibm.com>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Michal Hocko <mhocko@...nel.org>,
        Baolin Wang <baolin.wang@...ux.alibaba.com>,
        Brice Goglin <brice.goglin@...il.com>,
        Feng Tang <feng.tang@...el.com>,
        Jonathan Cameron <Jonathan.Cameron@...wei.com>
Subject: Re: RFC: Memory Tiering Kernel Interfaces

On 5/3/22 18:31, Wei Xu wrote:
>> Well, x86 CPUs have performance monitoring hardware that can
>> theoretically collect physical access information too.  But, this
>> performance monitoring hardware wasn't designed for this specific use
>> case in mind.  So, in practice, these events (PEBS) weren't very useful
>> for driving memory tiering.
> The PEBS events without any filtering might not be useful for memory
> tiering, but the PEBS events with hardware-based data source filtering
> can be useful in driving promotions in memory tiering. Certainly,
> because these events are not designed for this specific use case in
> mind, there are inefficiencies using them for memory tiering, e.g.
> instead of just getting a heat counter for each hot page, we can get
> events repeatedly on the hot pages.

Also, I believe the addresses that come out of the PEBS events are
virtual addresses (Data Linear Addresses according to the SDM).  If the
events are written from a KVM guest, you get guest linear addresses.

That means a lot of page table and EPT walks to map those linear
addresses back to physical.  That adds to the inefficiency.

In the end, you get big PEBS buffers with lots of irrelevant data that
needs significant post-processing to make sense of it.  The folks at
Intel that tried this really struggled to take this mess and turn it
into a successful hot-page tracking.

Maybe someone else will find a better way to do it, but we tried and
gave up.