linux-kernel - [RFC v2 0/5] Transparent on-demand struct page initialization embedded in the buddy allocator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1375465467-40488-1-git-send-email-nzimmer@sgi.com>
Date:	Fri,  2 Aug 2013 12:44:22 -0500
From:	Nathan Zimmer <nzimmer@....com>
To:	hpa@...or.com, mingo@...nel.org
Cc:	linux-kernel@...r.kernel.org, linux-mm@...ck.org, holt@....com,
	nzimmer@....com, rob@...dley.net, travis@....com,
	daniel@...ascale-asia.com, akpm@...ux-foundation.org,
	gregkh@...uxfoundation.org, yinghai@...nel.org, mgorman@...e.de
Subject: [RFC v2 0/5] Transparent on-demand struct page initialization embedded in the buddy allocator

We are still restricting ourselves ourselves to 2MiB initialization to keep the
patch set a little smaller and more clear.

We are still struggling with the expand().  Nearly always the first reference
to a struct page which is in the middle of the 2MiB region.  We were unable to
find a good solution.  Also, given the strong warning at the head of expand(),
we did not feel experienced enough to refactor it to make things always
reference the 2MiB page first.
The only other fastpath impact left is the expansion in prep_new_page.

With this patch, we did boot a 16TiB machine.
The two main areas that benefit from this patch is free_all_bootmem and
memmap_init_zone.  Without the patches it took 407 seconds and 1151 seconds
respectively.  With the patches it took 220 and 49 seconds respectively.
This is a total savings of 1289 seconds (21 minutes).
These times were aquired using a modified version of script which record the
time in uSecs at the beginning of each line of output.

The previous patch set was faster through free_all_bootmem but I wanted to
include Yinghai suggestion.  Hopefully I didn't miss the mark too much with
that patch and yes I do still need to optimize it.

I know there are some still rough parts but I wanted to check in with the full
patch set.

Nathan Zimmer (1):
  Only set page reserved in the memblock region

Robin Holt (4):
  memblock: Introduce a for_each_reserved_mem_region iterator.
  Have __free_pages_memory() free in larger chunks.
  Move page initialization into a separate function.
  Sparse initialization of struct page array.

 include/linux/memblock.h   |  18 +++++
 include/linux/mm.h         |   2 +
 include/linux/page-flags.h |   5 +-
 mm/memblock.c              |  32 ++++++++
 mm/mm_init.c               |   2 +-
 mm/nobootmem.c             |  28 +++----
 mm/page_alloc.c            | 194 ++++++++++++++++++++++++++++++++++++---------
 7 files changed, 225 insertions(+), 56 deletions(-)

-- 
1.8.2.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/