linux-kernel - Re: [RFC PATCH] mm: CONFIG_NR_ZONES

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 29 Feb 2016 09:55:17 -0800
From:	Dan Williams <dan.j.williams@...el.com>
To:	Vlastimil Babka <vbabka@...e.cz>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Rik van Riel <riel@...hat.com>,
	Dave Hansen <dave.hansen@...ux.intel.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	Linux MM <linux-mm@...ck.org>, Mel Gorman <mgorman@...e.de>,
	Mark <markk@...ra.co.uk>, Joonsoo Kim <iamjoonsoo.kim@....com>,
	Sudip Mukherjee <sudipm.mukherjee@...il.com>
Subject: Re: [RFC PATCH] mm: CONFIG_NR_ZONES_EXTENDED

On Mon, Feb 29, 2016 at 4:33 AM, Vlastimil Babka <vbabka@...e.cz> wrote:
> On 02/02/2016 06:42 AM, Andrew Morton wrote:
>>
>> On Wed, 27 Jan 2016 22:19:14 -0800 Dan Williams <dan.j.williams@...el.com>
>> wrote:
>>
>>> ZONE_DEVICE (merged in 4.3) and ZONE_CMA (proposed) are examples of new
>>> mm zones that are bumping up against the current maximum limit of 4
>>> zones, i.e. 2 bits in page->flags.  When adding a zone this equation
>>> still needs to be satisified:
>>>
>>>      SECTIONS_WIDTH + ZONES_WIDTH + NODES_SHIFT + LAST_CPUPID_SHIFT
>>>           <= BITS_PER_LONG - NR_PAGEFLAGS
>>>
>>> ZONE_DEVICE currently tries to satisfy this equation by requiring that
>>> ZONE_DMA be disabled, but this is untenable given generic kernels want
>>> to support ZONE_DEVICE and ZONE_DMA simultaneously.  ZONE_CMA would like
>>> to increase the amount of memory covered per section, but that limits
>>> the minimum granularity at which consecutive memory ranges can be added
>>> via devm_memremap_pages().
>>>
>>> The trade-off of what is acceptable to sacrifice depends heavily on the
>>> platform.  For example, ZONE_CMA is targeted for 32-bit platforms where
>>> page->flags is constrained, but those platforms likely do not care about
>>> the minimum granularity of memory hotplug.  A big iron machine with 1024
>>> numa nodes can likely sacrifice ZONE_DMA where a general purpose
>>> distribution kernel can not.
>>>
>>> CONFIG_NR_ZONES_EXTENDED is a configuration symbol that gets selected
>>> when the number of configured zones exceeds 4.  It documents the
>>> configuration symbols and definitions that get modified when ZONES_WIDTH
>>> is greater than 2.
>>>
>>> For now, it steals a bit from NODES_SHIFT.  Later on it can be used to
>>> document the definitions that get modified when a 32-bit configuration
>>> wants more zone bits.
>>
>>
>> So if you want ZONE_DMA, you're limited to 512 NUMA nodes?
>>
>> That seems reasonable.
>
>
> Sorry for the late reply, but it seems that with !SPARSEMEM, or with
> SPARSEMEM_VMEMMAP, reducing NUMA nodes isn't even necessary, because
> SECTIONS_WIDTH is zero (see the diagrams in linux/page-flags-layout.h). In
> my brief tests with 4.4 based kernel with SPARSEMEM_VMEMMAP it seems that
> with 1024 NUMA nodes and 8192 CPU's, there's still 7 bits left (i.e. 6 with
> CONFIG_NR_ZONES_EXTENDED).
>
> With the danger of becoming even more complex, could the limit also depend
> on CONFIG_SPARSEMEM/VMEMMAP to reflect that somehow?

In this case it's already part of the equation because:

config ZONE_DEVICE
       depends on MEMORY_HOTPLUG
       depends on MEMORY_HOTREMOVE

...and those in turn depend on SPARSEMEM.