linux-kernel - Re: [PATCH V3 0/4] Define coherent device memory node

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 6 Mar 2017 11:18:23 +0530
From:   Anshuman Khandual <khandual@...ux.vnet.ibm.com>
To:     Michal Hocko <mhocko@...nel.org>,
        Jerome Glisse <jglisse@...hat.com>
Cc:     Anshuman Khandual <khandual@...ux.vnet.ibm.com>,
        Mel Gorman <mgorman@...e.de>, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org, vbabka@...e.cz, minchan@...nel.org,
        aneesh.kumar@...ux.vnet.ibm.com, bsingharora@...il.com,
        srikar@...ux.vnet.ibm.com, haren@...ux.vnet.ibm.com,
        dave.hansen@...el.com, dan.j.williams@...el.com
Subject: Re: [PATCH V3 0/4] Define coherent device memory node

On 02/22/2017 10:24 PM, Michal Hocko wrote:
> On Wed 22-02-17 09:59:15, Jerome Glisse wrote:
>> On Wed, Feb 22, 2017 at 10:29:21AM +0100, Michal Hocko wrote:
>>> On Tue 21-02-17 18:39:17, Anshuman Khandual wrote:
>>>> On 02/17/2017 07:02 PM, Mel Gorman wrote:
>>
>> [...]
>>
>>> [...]
>>>> These are the reasons which prohibit the use of HMM for coherent
>>>> addressable device memory purpose.
>>>>
>>> [...]
>>>> (3) Application cannot directly allocate into device memory from user
>>>> space using existing memory related system calls like mmap() and mbind()
>>>> as the device memory hides away in ZONE_DEVICE.
>>>
>>> Why cannot the application simply use mmap on the device file?
>>
>> This has been said before but we want to share the address space this do
>> imply that you can not rely on special allocator. For instance you can
>> have an application that use a library and the library use the GPU but
>> the application is un-aware and those any data provided by the application
>> to the library will come from generic malloc (mmap anonymous or from
>> regular file).
>>
>> Currently what happens is that the library reallocate memory through
>> special allocator and copy thing. Not only does this waste memory (the
>> new memory is often regular memory too) but you also have to paid the
>> cost of copying GB of data.
>>
>> Last bullet to this, is complex data structure (list, tree, ...) having
>> to go through special allocator means you have re-build the whole structure
>> with the duplicated memory.
>>
>>
>> Allowing to directly use memory allocated from malloc (mmap anonymous
>> private or from a regular file) avoid the copy operation and the complex
>> duplication of data structure. Moving the dataset to the GPU is then a
>> simple memory migration from kernel point of view.
>>
>> This is share address space without special allocator is mandatory in new
>> or future standard such as OpenCL, Cuda, C++, OpenMP, ... some other OS
>> already have this and the industry want it. So the questions is do we
>> want to support any of this, do we care about GPGPU ?
>>
>>
>> I believe we want to support all this new standard but maybe i am the
>> only one.
>>
>> In HMM case i have the extra painfull fact that the device memory is
>> not accessible by the CPU. For CDM on contrary, CPU can access in a
>> cache coherent way the device memory and all operation behave as regular
>> memory (thing like atomic operation for instance).
>>
>>
>> I hope this clearly explain why we can no longer rely on dedicated/
>> specialized memory allocator.
> 
> Yes this clarifies this point. Thanks for the information which would be
> really helpful in the initial description. Maybe I've just missed it,
> though.

Sure, will add this into the patch description.