linux-kernel - [LSF/MM TOPIC] NUMA, memory hierarchy and device memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 18 Jan 2019 12:45:13 -0500
From:   Jerome Glisse <jglisse@...hat.com>
To:     lsf-pc@...ts.linux-foundation.org
Cc:     linux-mm@...ck.org, Dan Williams <dan.j.williams@...el.com>,
        Dave Hansen <dave.hansen@...el.com>,
        Felix Kuehling <Felix.Kuehling@....com>,
        John Hubbard <jhubbard@...dia.com>,
        Jonathan Cameron <jonathan.cameron@...wei.com>,
        Keith Busch <keith.busch@...el.com>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Michal Hocko <mhocko@...nel.org>,
        Paul Blinzer <Paul.Blinzer@....com>,
        linux-kernel@...r.kernel.org
Subject: [LSF/MM TOPIC] NUMA, memory hierarchy and device memory

Hi, i would like to discuss about NUMA API and its short comings when
it comes to memory hierarchy (from fast HBM, to slower persistent
memory through regular memory) and also device memory (which can have
its own hierarchy).

I have proposed a patch to add a new memory topology model to the
kernel for application to be able to get that informations, it
also included a set of new API to bind/migrate process range [1].
Note that this model also support device memory.

So far device memory support is achieve through device specific ioctl
and this forbid some scenario like device memory interleaving accross
multiple devices for a range. It also make the whole userspace more
complex as program have to mix and match multiple device specific API
on top of NUMA API.

While memory hierarchy can be more or less expose through the existing
NUMA API by creating node for non-regular memory [2], i do not see this
as a satisfying solution. Moreover such scheme does not work for device
memory that might not even be accessible by CPUs.

Hence i would like to discuss few points:
    - What proof people wants to see this as problem we need to solve ?
    - How to build concensus to move forward on this ?
    - What kind of syscall API people would like to see ?

People to discuss this topic:
    Dan Williams <dan.j.williams@...el.com>
    Dave Hansen <dave.hansen@...el.com>
    Felix Kuehling <Felix.Kuehling@....com>
    John Hubbard <jhubbard@...dia.com>
    Jonathan Cameron <jonathan.cameron@...wei.com>
    Keith Busch <keith.busch@...el.com>
    Mel Gorman <mgorman@...hsingularity.net>
    Michal Hocko <mhocko@...nel.org>
    Paul Blinzer <Paul.Blinzer@....com>

Probably others, sorry if i miss anyone from previous discussions.

Cheers,
Jérôme

[1] https://lkml.org/lkml/2018/12/3/1072
[2] https://lkml.org/lkml/2018/12/10/1112