linux-kernel - Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150512094709.GB11477@gmail.com>
Date:	Tue, 12 May 2015 11:47:09 +0200
From:	Ingo Molnar <mingo@...nel.org>
To:	Len Brown <lenb@...nel.org>
Cc:	Jan H. Schönherr <jschoenh@...zon.de>,
	Thomas Gleixner <tglx@...utronix.de>, X86 ML <x86@...nel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	Anthony Liguori <aliguori@...zon.com>,
	Ingo Molnar <mingo@...hat.com>,
	"H. Peter Anvin" <hpa@...or.com>, Tim Deegan <tim@....org>,
	Gang Wei <gang.wei@...el.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Arjan van de Ven <arjan@...radead.org>
Subject: Re: [PATCH] x86: skip delays during SMP initialization similar to Xen


* Len Brown <lenb@...nel.org> wrote:

> > It was 20+ seconds before that, 10+ seconds for the SMP bootup 
> > sequence.
> 
> (1.625928-0.558947) = 1.07 seconds to online 119 additional cpus.
> /119 = .0089662268 each, lets call it 9ms.
> 
> Here is my ivb-ex running stock fedora 21's Linux-3.19 (no patch applied):
> 
> [    0.404369] x86: Booting SMP configuration:
> [    0.409492] .... node  #0, CPUs:          #1
> [    0.439900] NMI watchdog: enabled on all CPUs, permanently consumes
> one hw-PMU counter.
> [    0.450533]    #2   #3   #4   #5   #6   #7   #8   #9  #10  #11  #12  #13  #14
> [    0.652409] .... node  #1, CPUs:    #15  #16  #17  #18  #19  #20
> #21  #22  #23  #24  #25  #26  #27  #28  #29
> [    0.999645] .... node  #2, CPUs:    #30  #31  #32  #33  #34  #35
> #36  #37  #38  #39  #40  #41  #42  #43  #44
> [    1.346991] .... node  #3, CPUs:    #45  #46  #47  #48  #49  #50
> #51  #52  #53  #54  #55  #56  #57  #58  #59
> [    1.694171] .... node  #0, CPUs:    #60  #61  #62  #63  #64  #65
> #66  #67  #68  #69  #70  #71  #72  #73  #74
> [    1.928248] .... node  #1, CPUs:    #75  #76  #77  #78  #79  #80
> #81  #82  #83  #84  #85  #86  #87  #88  #89
> [    2.198370] .... node  #2, CPUs:    #90  #91  #92  #93  #94  #95
> #96  #97  #98  #99 #100 #101 #102 #103 #104
> [    2.468574] .... node  #3, CPUs:   #105 #106 #107 #108 #109 #110
> #111 #112 #113 #114 #115 #116 #117 #118 #119
> [    2.737884] x86: Booted up 4 nodes, 120 CPUs
> [    2.743758] smpboot: Total of 120 processors activated (671097.18 BogoMIPS)
> 
> (2.743758-0.404369) = 2.339389 for all 119 processors
> /119 = .01965873109243697478 - lets call it 19ms each
> 
> so this baseline case is 19ms/processor -- which matches above, where
> we delete 10ms/processor.
> But even at 19ms each, this is totals only 2.3 seconds for all 119 processors.
> So I don't understand your reference to 10+ seconds, above.

So I was booting 120 CPUs with kvmtool (tools/kvm/ under -tip).

Even with your patches applied it's 7 seconds (config attached):

[    0.152189] x86: Booting SMP configuration:
[    0.152705] .... node  #0, CPUs:          #1   #2   #3   #4   #5   #6   #7   #8   #9  #10  #11  #12  #13  #14  #15  #16  #17  #18  #19  #20  #21  #22  #23  #24  #25  #26  #27  #28  #29  #30  #31  #32  #33  #34  #35  #36  #37  #38  #39  #40  #41  #42  #43  #44  #45  #46  #47  #48  #49  #50  #51  #52  #53  #54  #55  #56  #57  #58  #59  #60  #61  #62  #63  #64  #65  #66  #67  #68  #69  #70  #71  #72  #73  #74  #75  #76  #77  #78  #79  #80  #81  #82  #83  #84  #85  #86  #87  #88  #89  #90  #91  #92  #93  #94  #95  #96  #97  #98  #99 #100 #101 #102 #103 #104 #105 #106 #107 #108 #109 #110 #111 #112 #113 #114 #115 #116 #117 #118 #119
[    7.627192] x86: Booted up 1 node, 120 CPUs
[    7.627795] smpboot: Total of 120 processors activated (672119.13 BogoMIPS)
[    7.633325] devtmpfs: initialized

so there's some other delay going on. It could very well be a kvmtool 
related initialization delay?

Thanks,

	Ingo

View attachment "config" of type "text/plain" (98726 bytes)