linux-kernel - Re: [RFC V2 03/12] mm: Change generic FALLBACK zonelist creation process

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9c951c50-3d75-2356-3f21-434ddca63f1b@nvidia.com>
Date:   Tue, 31 Jan 2017 13:14:59 -0600
From:   David Nellans <dnellans@...dia.com>
To:     Dave Hansen <dave.hansen@...el.com>,
        John Hubbard <jhubbard@...dia.com>,
        Anshuman Khandual <khandual@...ux.vnet.ibm.com>,
        <linux-kernel@...r.kernel.org>, <linux-mm@...ck.org>
CC:     <mhocko@...e.com>, <vbabka@...e.cz>, <mgorman@...e.de>,
        <minchan@...nel.org>, <aneesh.kumar@...ux.vnet.ibm.com>,
        <bsingharora@...il.com>, <srikar@...ux.vnet.ibm.com>,
        <haren@...ux.vnet.ibm.com>, <jglisse@...hat.com>,
        <dan.j.williams@...el.com>
Subject: Re: [RFC V2 03/12] mm: Change generic FALLBACK zonelist creation
 process



On 01/31/2017 12:04 PM, Dave Hansen wrote:
> On 01/30/2017 11:25 PM, John Hubbard wrote:
>> I also don't like having these policies hard-coded, and your 100x
>> example above helps clarify what can go wrong about it. It would be
>> nicer if, instead, we could better express the "distance" between nodes
>> (bandwidth, latency, relative to sysmem, perhaps), and let the NUMA
>> system figure out the Right Thing To Do.
>>
>> I realize that this is not quite possible with NUMA just yet, but I
>> wonder if that's a reasonable direction to go with this?
> In the end, I don't think the kernel can make the "right" decision very
> widely here.
>
> Intel's Xeon Phis have some high-bandwidth memory (MCDRAM) that
> evidently has a higher latency than DRAM.  Given a plain malloc(), how
> is the kernel to know that the memory will be used for AVX-512
> instructions that need lots of bandwidth vs. some random data structure
> that's latency-sensitive?
>
> In the end, I think all we can do is keep the kernel's existing default
> of "low latency to the CPU that allocated it", and let apps override
> when that policy doesn't fit them.
>
I think John's point is that latency might not be the predominant factor
anymore
for certain sections of the CPU and GPU world.  What if a Phi has MCDRAM
physically attached, but DDR4 connected via QPI that still has lower
total latency (might
be a stretch for Phi but not a stretch for GPUs with deep sorting memory
controllers)?  Lowest latency is probably the wrong choice. Latency has
really been a
numeric proxy for physical proximity, under assumption most closely
coupled memory is
the right placement, but HBM/MCDRAM is causing that relationship to
break down in all
sorts of interesting ways.