linux-kernel - Re: [PATCH v5 0/9] mm/demotion: Memory tiers and demotion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YqCq0cUCnQmS6SV4@cmpxchg.org>
Date:   Wed, 8 Jun 2022 09:57:37 -0400
From:   Johannes Weiner <hannes@...xchg.org>
To:     "Aneesh Kumar K.V" <aneesh.kumar@...ux.ibm.com>
Cc:     linux-mm@...ck.org, akpm@...ux-foundation.org,
        Wei Xu <weixugc@...gle.com>, Huang Ying <ying.huang@...el.com>,
        Greg Thelen <gthelen@...gle.com>,
        Yang Shi <shy828301@...il.com>,
        Davidlohr Bueso <dave@...olabs.net>,
        Tim C Chen <tim.c.chen@...el.com>,
        Brice Goglin <brice.goglin@...il.com>,
        Michal Hocko <mhocko@...nel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Hesham Almatary <hesham.almatary@...wei.com>,
        Dave Hansen <dave.hansen@...el.com>,
        Jonathan Cameron <Jonathan.Cameron@...wei.com>,
        Alistair Popple <apopple@...dia.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Feng Tang <feng.tang@...el.com>,
        Jagdish Gediya <jvgediya@...ux.ibm.com>,
        Baolin Wang <baolin.wang@...ux.alibaba.com>,
        David Rientjes <rientjes@...gle.com>
Subject: Re: [PATCH v5 0/9] mm/demotion: Memory tiers and demotion

Hi Aneesh,

On Fri, Jun 03, 2022 at 07:12:28PM +0530, Aneesh Kumar K.V wrote:
> * The current tier initialization code always initializes
>   each memory-only NUMA node into a lower tier.  But a memory-only
>   NUMA node may have a high performance memory device (e.g. a DRAM
>   device attached via CXL.mem or a DRAM-backed memory-only node on
>   a virtual machine) and should be put into a higher tier.

I have to disagree with this premise. The CXL.mem bus has different
latency and bandwidth characteristics. It's also conceivable that
cheaper and slower DRAM is connected to the CXL bus (think recycling
DDR4 DIMMS after switching to DDR5). DRAM != DRAM.

Our experiments with production workloads show regressions between
15-30% in serviced requests when you don't distinguish toptier DRAM
from lower tier DRAM. While it's fixable with manual tuning, your
patches would bring reintroduce this regression it seems.

Making tiers explicit is a good idea, but can we keep the current
default that CPU-less nodes are of a lower tier than ones with CPU?
I'm having a hard time imagining where this wouldn't be true... Or why
it shouldn't be those esoteric cases that need the manual tuning.