linux-kernel - Re: Do we need to call calibrate_delay() for all cores/hyperthreads on a socket?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20101203093617.GB8800@basil.fritz.box>
Date:	Fri, 3 Dec 2010 10:36:18 +0100
From:	Andi Kleen <andi@...stfloor.org>
To:	Robin Holt <holt@....com>
Cc:	linux-kernel@...r.kernel.org,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Adrian Bunk <bunk@...nel.org>,
	Venkatesh Pallipadi <venkatesh.pallipadi@...el.com>,
	Andi Kleen <andi@...stfloor.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>,
	"H. Peter Anvin" <hpa@...or.com>, Jack Steiner <steiner@....com>
Subject: Re: Do we need to call calibrate_delay() for all
 cores/hyperthreads on a socket?

On Fri, Dec 03, 2010 at 12:52:12AM -0600, Robin Holt wrote:
> 
> I do not know all the different combinations of sockets, cores, and
> hyperthreads out there, but it seems like all cores and their hyperthreads
> on a socket should compute the same value for their calculate_delay()
> function.
> 
> When booting a 4096 cpu system without specifying lpj on the command line,
> we spend approximately 0.1 seconds per core/hyperthread calculating the
> lpj value for that cpu.
> 
> If we were to, on the other hand, only calculate the delay value for the
> first core on a socket, we would reduce the time spent booting a 4096 cpu
> (256 sockets, 8 cores per socket hyperthreaded) down from nearly seven
> minutes to approx 25 seconds.  This seems like a very safe optimization,
> but I repeat that I do not know all the different potential combinations
> of socket, core, hyperthread out that.  Please note these are just rough
> approximations taken from memory.  I am doing a couple of test boots
> now without and with lpj= specified on the command line to get a more
> accurate approximation.

Hi Robin,

At least for SMT threads that seems like a fine idea: in general I think
we should be doing a lot more things "per core" instead of "per thread".

As for sharing calibration on whole sockets: on some systems individual 
cores on a socket can run with different frequencies (e.g. AMD Fam10h).
In theory the CPU should always run at P0 when coming out of boot,
but we've had cases in the past where it was running lower, e.g.
if the BIOS decided to cap power consumption. I guess in theory
those states could be different per core on those systems.

So I think it would be safer to still calibrate per core at least.
In theory one could try to check for the case described above,
and only do it then, but that may be fragile.

But maybe it's possible to calibrate different cores in a socket in 
parallel anyways?

-Andi

-- 
ak@...ux.intel.com -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/