lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <51C4C356.3040602@sgi.com>
Date:	Fri, 21 Jun 2013 14:19:18 -0700
From:	Mike Travis <travis@....com>
To:	Yinghai Lu <yinghai@...nel.org>
CC:	Nathan Zimmer <nzimmer@....com>, Robin Holt <holt@....com>,
	Rob Landley <rob@...dley.net>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>,
	"H. Peter Anvin" <hpa@...or.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	the arch/x86 maintainers <x86@...nel.org>,
	linux-doc@...r.kernel.org,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [RFC 0/2] Delay initializing of large sections of memory



On 6/21/2013 11:36 AM, Yinghai Lu wrote:
> On Fri, Jun 21, 2013 at 9:25 AM, Nathan Zimmer <nzimmer@....com> wrote:
>> This rfc patch set delays initializing large sections of memory until we have
>> started cpus.  This has the effect of reducing startup times on large memory
>> systems.  On 16TB it can take over an hour to boot and most of that time
>> is spent initializing memory.
> 
> One hour on system with 16T ram? BIOS or OS?

The BIOS is about 20 minutes *before* the 1+ hour.  (When we started this
UV project way back when, 8TB took over 4 hours before we threw in the towel.)
> 
> I use wall clock to check bootime on one system with 3T and 16 pcie cards,
> Linus only takes about 3m and 30 seconds from bootloader.

I can send some stats on where various delays are but most of it was in
memory initialization.  On average UV nodes carry 128 or 256G per node,
so 12 nodes would take about 3 or 4 minutes, perhaps more.
> 
> wonder if you boot delay is with so many cpu get onlined in serialized mode.

Nope.  If that was the case, delaying memory but initializing all the
cpus would not affect the time.
> 
> so can you try boot your system with "maxcpus=128" to get the boot time with
> wall clock ?

We could try if you really think it will provide any useful info.  Not sure
exactly how having memory on nodes with no active cpus will react.
> 
> Thanks
> 
> Yinghai
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ