linux-kernel - Re: [PATCH v14 04/10] mm/demotion/dax/kmem: Set node's abstract distance to MEMTIER_DEFAULT_DAX

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <cd1c13ee-6fc3-bde8-96f9-8c3c93441275@linux.ibm.com>
Date:   Tue, 16 Aug 2022 10:39:44 +0530
From:   Aneesh Kumar K V <aneesh.kumar@...ux.ibm.com>
To:     "Huang, Ying" <ying.huang@...el.com>
Cc:     linux-mm@...ck.org, akpm@...ux-foundation.org,
        Wei Xu <weixugc@...gle.com>, Yang Shi <shy828301@...il.com>,
        Davidlohr Bueso <dave@...olabs.net>,
        Tim C Chen <tim.c.chen@...el.com>,
        Michal Hocko <mhocko@...nel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Hesham Almatary <hesham.almatary@...wei.com>,
        Dave Hansen <dave.hansen@...el.com>,
        Jonathan Cameron <Jonathan.Cameron@...wei.com>,
        Alistair Popple <apopple@...dia.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Johannes Weiner <hannes@...xchg.org>, jvgediya.oss@...il.com,
        Bharata B Rao <bharata@....com>
Subject: Re: [PATCH v14 04/10] mm/demotion/dax/kmem: Set node's abstract
 distance to MEMTIER_DEFAULT_DAX_ADISTANCE

On 8/15/22 8:09 AM, Huang, Ying wrote:
> "Aneesh Kumar K.V" <aneesh.kumar@...ux.ibm.com> writes:
> 
>> By default, all nodes are assigned to the default memory tier which
>> is the memory tier designated for nodes with DRAM
>>
>> Set dax kmem device node's tier to slower memory tier by assigning
>> abstract distance to MEMTIER_DEFAULT_DAX_ADISTANCE. Low-level drivers
>> like papr_scm or ACPI NFIT can initialize memory device type to a
>> more accurate value based on device tree details or HMAT. If the
>> kernel doesn't find the memory type initialized, a default slower
>> memory type is assigned by the kmem driver.
>>
>> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@...ux.ibm.com>
>> ---
>>  drivers/dax/kmem.c           | 42 +++++++++++++++--
>>  include/linux/memory-tiers.h | 42 ++++++++++++++++-
>>  mm/memory-tiers.c            | 91 +++++++++++++++++++++++++++---------
>>  3 files changed, 149 insertions(+), 26 deletions(-)
>>
>> diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
>> index a37622060fff..d88814f1c414 100644
>> --- a/drivers/dax/kmem.c
>> +++ b/drivers/dax/kmem.c
>> @@ -11,9 +11,17 @@
>>  #include <linux/fs.h>
>>  #include <linux/mm.h>
>>  #include <linux/mman.h>
>> +#include <linux/memory-tiers.h>
>>  #include "dax-private.h"
>>  #include "bus.h"
>>  
>> +/*
>> + * Default abstract distance assigned to the NUMA node onlined
>> + * by DAX/kmem if the low level platform driver didn't initialize
>> + * one for this NUMA node.
>> + */
>> +#define MEMTIER_DEFAULT_DAX_ADISTANCE	(MEMTIER_ADISTANCE_DRAM * 2)
> 
> If my understanding were correct, this is targeting Optane DCPMM for
> now.  The measured results in the following paper is,
> 
> https://arxiv.org/pdf/2002.06018.pdf
> 
> Section: 2.1 Read/Write Latencies
> 
> "
> For read access, the latency of DCPMM was 400.1% higher than that of
> DRAM. For write access, it was 407.1% higher.
> "
> 
> Section: 2.2 Read/Write Bandwidths
> 
> "
> For read access, the throughput of DCPMM was 37.1% of DRAM. For write
> access, it was 7.8%
> "
> 
> According to the above data, I think the MEMTIER_DEFAULT_DAX_ADISTANCE
> can be "5 * MEMTIER_ADISTANCE_DRAM".
> 

If we look at mapping every 100% increase in latency as a memory tier, we essentially
will have 4 memory tier here. Each memory tier is covering a range of abstract distance 128.
which makes a total adistance increase from MEMTIER_ADISTANCE_DRAM by 512. This puts 
DEFAULT_DAX_DISTANCE at 1024 or  MEMTIER_ADISTANCE_DRAM * 2

-aneesh