linux-kernel - [Patch] stop_machine stalls for a considerable period on large cpu count machines.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20090627105711.GB6894@sgi.com>
Date:	Sat, 27 Jun 2009 05:57:11 -0500
From:	Robin Holt <holt@....com>
To:	linux-kernel@...r.kernel.org
Subject: [Patch] stop_machine stalls for a considerable period on large cpu
	count machines.

I forgot to include the linux-kernel mailing list.

Robin

----- Forwarded message from Robin Holt <holt@....com> -----

Date: Sat, 27 Jun 2009 05:53:11 -0500
From: Robin Holt <holt@....com>
To: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Rusty Russell <rusty@...tcorp.com.au>, stable@...nel.org,
	mike travis <travis@....com>
Subject: [Patch] stop_machine stalls for a considerable period on large cpu
	count machines.

Mike Travis noted that a 2048 cpu machine booting would take hours
to get through its modprobes.  We would get numerous back traces from
stop_cpu indicating they had not serviced interrupts.

A quick code review indicated we have a situation of heavy cacheline                                                      contention due to the 'state' (read-mostly) and 'thread_ack'
(write-mostly) variables being located in the same cacheline.


Signed-off-by: Robin Holt <holt@....com>

---
 kernel/stop_machine.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Index: stop_machine_false_sharing/kernel/stop_machine.c
===================================================================
--- stop_machine_false_sharing.orig/kernel/stop_machine.c	2009-06-27 05:26:57.000000000 -0500
+++ stop_machine_false_sharing/kernel/stop_machine.c	2009-06-27 05:34:45.182637682 -0500
@@ -36,8 +36,8 @@ struct stop_machine_data {
 
 /* Like num_online_cpus(), but hotplug cpu uses us, so we need this. */
 static unsigned int num_threads;
-static atomic_t thread_ack;
-static DEFINE_MUTEX(lock);
+static atomic_t thread_ack	____cacheline_aligned;
+static DEFINE_MUTEX(lock)	____cacheline_aligned;
 /* setup_lock protects refcount, stop_machine_wq and stop_machine_work. */
 static DEFINE_MUTEX(setup_lock);
 /* Users of stop_machine. */

----- End forwarded message -----
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/