Message-Id: <13c1a501-0ab9-898c-f749-efecca787661@linux.vnet.ibm.com>
Date:   Tue, 14 Mar 2017 19:03:04 +0530
From:   Anshuman Khandual <khandual@...ux.vnet.ibm.com>
To:     John Hubbard <jhubbard@...dia.com>,
        Anshuman Khandual <khandual@...ux.vnet.ibm.com>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org
Cc:     mhocko@...e.com, vbabka@...e.cz, mgorman@...e.de,
        minchan@...nel.org, aneesh.kumar@...ux.vnet.ibm.com,
        bsingharora@...il.com, srikar@...ux.vnet.ibm.com,
        haren@...ux.vnet.ibm.com, jglisse@...hat.com,
        dave.hansen@...el.com, dan.j.williams@...el.com,
        zi.yan@...rutgers.edu
Subject: Re: [PATCH 1/2] mm: Change generic FALLBACK zonelist creation process

On 03/08/2017 04:37 PM, John Hubbard wrote:
> On 03/08/2017 01:21 AM, Anshuman Khandual wrote:
>> Kernel allocation to CDM node has already been prevented by putting it's
>> entire memory in ZONE_MOVABLE. But the CDM nodes must also be isolated
>> from implicit allocations happening on the system.
>>
>> Any isolation seeking CDM node requires isolation from implicit memory
>> allocations from user space but at the same time there should also have
>> an explicit way to do the memory allocation.
>>
>> Platform node's both zonelists are fundamental to where the memory comes
>> from when there is an allocation request. In order to achieve these two
>> objectives as stated above, zonelists building process has to change as
>> both zonelists (i.e FALLBACK and NOFALLBACK) gives access to the node's
>> memory zones during any kind of memory allocation. The following changes
>> are implemented in this regard.
>>
>> * CDM node's zones are not part of any other node's FALLBACK zonelist
>> * CDM node's FALLBACK list contains it's own memory zones followed by
>>   all system RAM zones in regular order as before


> 
> There was a discussion, on an earlier version of this patchset, in which
> someone pointed out that a slight over-allocation on a device that has
> much more memory than the CPU has, could use up system memory. Your
> latest approach here does not address this.

Hmm, I dont remember this. Could you please be more specific and point
me to the discussion on this.

> 
> I'm thinking that, until oversubscription between NUMA nodes is more
> fully implemented in a way that can be properly controlled, you'd

I did not get you. What does over subscription mean in this context ?
FALLBACK zonelist on each node has memory from every node including
it's own. Hence the allocation request targeted towards any node is
symmetrical with respect to from where the memory will be allocated.

> probably better just not fallback to system memory. In other words, a
> CDM node really is *isolated* from other nodes--no automatic use in
> either direction.

That is debatable. With this proposed solution the CDM FALLBACK
zonelist contains system RAM zones as fallback option which will
be used in case CDM memory is depleted. IMHO, I think thats the
right thing to do as it still maintains the symmetry to some
extent.

> 
> Also, naming and purpose: maybe this is a "Limited NUMA Node", rather
> than a Coherent Device Memory node. Because: the real point of this
> thing is to limit the normal operation of NUMA, just enough to work with
> what I am *told* is memory-that-is-too-fragile-for-kernel-use (I remain
> soemwhat on the fence, there, even though you did talk me into it
> earlier, heh).

:) Naming can be debated later after we all agree on the proposal
in principle. We have already discussed about kernel memory on CDM
in detail.

> 
> On process: it would probably help if you gathered up previous
> discussion points and carefully, concisely addressed each one,
> somewhere, (maybe in a cover letter). Because otherwise, it's too easy
> for earlier, important problems to be forgotten. And reviewers don't
> want to have to repeat themselves, of course.

Will do.

Powered by blists - more mailing lists