linux-kernel - Re: Interacting with coherent memory on external devices

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <55390EE1.8020304@gmail.com>
Date:	Thu, 23 Apr 2015 11:25:21 -0400
From:	Austin S Hemmelgarn <ahferroin7@...il.com>
To:	Christoph Lameter <cl@...ux.com>,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>
CC:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Jerome Glisse <j.glisse@...il.com>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	jglisse@...hat.com, mgorman@...e.de, aarcange@...hat.com,
	riel@...hat.com, airlied@...hat.com,
	aneesh.kumar@...ux.vnet.ibm.com,
	Cameron Buschardt <cabuschardt@...dia.com>,
	Mark Hairgrove <mhairgrove@...dia.com>,
	Geoffrey Gerfin <ggerfin@...dia.com>,
	John McKenna <jmckenna@...dia.com>, akpm@...ux-foundation.org
Subject: Re: Interacting with coherent memory on external devices

On 2015-04-23 10:25, Christoph Lameter wrote:
> On Thu, 23 Apr 2015, Benjamin Herrenschmidt wrote:
>
>> They are via MMIO space. The big differences here are that via CAPI the
>> memory can be fully cachable and thus have the same characteristics as
>> normal memory from the processor point of view, and the device shares
>> the MMU with the host.
>>
>> Practically what that means is that the device memory *is* just some
>> normal system memory with a larger distance. The NUMA model is an
>> excellent representation of it.
>
> I sure wish you would be working on using these features to increase
> performance and the speed of communication to devices.
>
> Device memory is inherently different from main memory (otherwise the
> device would be using main memory) and thus not really NUMA. NUMA at least
> assumes that the basic characteristics of memory are the same while just
> the access speeds vary. GPU memory has very different performance
> characteristics and the various assumptions on memory that the kernel
> makes for the regular processors may not hold anymore.
>
You are restricting your definition of NUMA to what the industry 
constrains it to mean.  Based solely on the academic definition of a 
NUMA system, this _is_ NUMA.  In fact, based on the academic definition, 
all modern systems could be considered to be NUMA systems, with each 
level of cache representing a memory only node.

Looking at this whole conversation, all I see is two different views on 
how to present the asymmetric multiprocessing arrangements that have 
become commonplace in today's systems to userspace.  Your model favors 
performance, while CAPI favors simplicity for userspace.



Download attachment "smime.p7s" of type "application/pkcs7-signature" (2967 bytes)