linux-kernel - Re: [RFC PATCH 0/5] Memory access profiler(IBS) driven NUMA balancing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <369bef08-92cc-9b55-823f-1fe780532967@amd.com>
Date:   Thu, 9 Feb 2023 11:34:35 +0530
From:   Bharata B Rao <bharata@....com>
To:     Dave Hansen <dave.hansen@...el.com>,
        Peter Zijlstra <peterz@...radead.org>
Cc:     linux-kernel@...r.kernel.org, linux-mm@...ck.org, mgorman@...e.de,
        mingo@...hat.com, bp@...en8.de, dave.hansen@...ux.intel.com,
        x86@...nel.org, akpm@...ux-foundation.org, luto@...nel.org,
        tglx@...utronix.de, yue.li@...verge.com,
        Ravikumar.Bangoria@....com, ying.huang@...el.com
Subject: Re: [RFC PATCH 0/5] Memory access profiler(IBS) driven NUMA balancing

On 2/8/2023 11:42 PM, Dave Hansen wrote:
> On 2/8/23 10:03, Peter Zijlstra wrote:
>>> - Hardware provided access information could be very useful for driving
>>>   hot page promotion in tiered memory systems. Need to check if this
>>>   requires different tuning/heuristics apart from what NUMA balancing
>>>   already does.
>> I think Huang Ying looked at that from the Intel POV and I think the
>> conclusion was that it doesn't really work out. What you need is
>> frequency information, but the PMU doesn't really give you that. You
>> need to process a *ton* of PMU data in-kernel.
> 
> Yeah, there were two big problems.
> 
> First, IIRC, Intel PEBS at the time only gave guest virtual addresses in
> the PEBS records.  They had to be translated back to host addresses to
> be usable.  That was extra expensive.

Just to be clear, I am using IBS in host only and it can give both virtual
and physical address.

> 
> Second, it *did* take a lot of processing to turn raw memory accesses
> into actionable frequency data.  That meant that we started in a hole
> performance-wise and had to make *REALLY* good decisions about page
> migration to make up for it.

I touched upon the frequency aspect in reply to Peter, but please let
me know if I am missing something.

> 
> The performance data here don't look awful, but they don't seem to add
> up to a clear win either.  I'm having a hard time imagining who would
> turn this on and how widely it would get used in practice.

I am hopeful with more appropriate tuning of NUMA balancing logic to work
with hardware-provided access info (as against scan based NUMA hint faults),
we should be able to see a clear win. At least theoretically we wouldn't
have the overheads of address space scanning and hint faults handling.

Thanks for your inputs.

Regards,
Bharata.