linux-kernel - Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130722061339.GC3421@sgi.com>
Date:	Mon, 22 Jul 2013 01:13:39 -0500
From:	Robin Holt <holt@....com>
To:	Yinghai Lu <yinghai@...nel.org>
Cc:	Robin Holt <holt@....com>, Sam Ben <sam.bennn@...il.com>,
	"H. Peter Anvin" <hpa@...or.com>, Ingo Molnar <mingo@...nel.org>,
	Nate Zimmer <nzimmer@....com>,
	Linux Kernel <linux-kernel@...r.kernel.org>,
	Linux MM <linux-mm@...ck.org>, Rob Landley <rob@...dley.net>,
	Mike Travis <travis@....com>,
	Daniel J Blueman <daniel@...ascale-asia.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Greg KH <gregkh@...uxfoundation.org>,
	Mel Gorman <mgorman@...e.de>
Subject: Re: [RFC 0/4] Transparent on-demand struct page initialization
 embedded in the buddy allocator

On Fri, Jul 19, 2013 at 04:51:49PM -0700, Yinghai Lu wrote:
> On Wed, Jul 17, 2013 at 2:30 AM, Robin Holt <holt@....com> wrote:
> > On Wed, Jul 17, 2013 at 01:17:44PM +0800, Sam Ben wrote:
> >> >With this patch, we did boot a 16TiB machine.  Without the patches,
> >> >the v3.10 kernel with the same configuration took 407 seconds for
> >> >free_all_bootmem.  With the patches and operating on 2MiB pages instead
> >> >of 1GiB, it took 26 seconds so performance was improved.  I have no feel
> >> >for how the 1GiB chunk size will perform.
> >>
> >> How to test how much time spend on free_all_bootmem?
> >
> > We had put a pr_emerg at the beginning and end of free_all_bootmem and
> > then used a modified version of script which record the time in uSecs
> > at the beginning of each line of output.
> 
> used two patches, found 3TiB system will take 100s before slub is ready.
> 
> about three portions:
> 1. sparse vmemap buf allocation, it is with bootmem wrapper, so clear those
> struct page area take about 30s.
> 2. memmap_init_zone: take about 25s
> 3. mem_init/free_all_bootmem about 30s.
> 
> so still wonder why 16TiB will need hours.

I don't know where you got the figure of hours for memory initialization.
That is likely for a 32TiB boot and includes the entire boot, not just
getting the memory allocator initialized.

For a 16 TiB boot:
1) 344
2) 1151
3) 407

I hope that illustrates why we chose to address the memmap_init_zone first
which had the nice side effect of also impacting the free_all_bootmem
slowdown.

With these patches, those numbers are currently:
1) 344
2) 49
3) 26

> also your patches looks like only address 2 and 3.

Right, but I thought that was the normal way to do things.  Address
one thing at a time and work toward a better kernel.  I don't see a
relationship between the work we are doing here and the sparse vmemmap
buffer allocation.  Have I missed something?

Did you happen to time a boot with these patches applied to see how
long it took and how much impact they had on a smaller config?

Robin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/