Message-Id: <20080624093452.946878437@bull.net>
Date:	Tue, 24 Jun 2008 11:34:52 +0200
From:	<Solofo.Ramangalahy@...l.net>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	<linux-kernel@...r.kernel.org>, Matt Helsley <matthltc@...ibm.com>,
	Mingming Cao <cmm@...ibm.com>,
	Nadia Derbey <Nadia.Derbey@...l.net>,
	Manfred Spraul <manfred@...orfullife.com>,
	Nick Piggin <nickpiggin@...oo.com.au>
Subject: [PATCH -mm 0/3] sysv ipc: increase msgmnb with the number of cpus

The size in bytes of a SysV IPC message queue, msgmnb, is too small
for large machines, but we don't want to bloat small machines.

This series change ("scale") the default value of
/proc/sys/kernel/msgmnb.

Several methods are used already to modify (mainly increase) msgmnb:
. distribution specific patch (e.g. openSUSE)
. system wide sysctl.conf
. application specific tuning via /proc/sys/kernel/msgmnb

Integrating this series would:
. reflect hardware and software evolutions and diversity,
. reduce configuration/tuning for the applications.

Here is the timeline of the evolution of MSG* #defines:
Year             1994         1999         1999          2008
Version           1.0       2.3.27       2.3.30        2.6.24
#define MSGMNI    128          128           16            16
#define MSGMAX   4056         8192         8192          8192
#define MSGMNB  16384        16384        16384         16384

This patch series increases msgmnb with respect to the number of
cpus/cores for larger machines. For uniprocessor machines the value
does not increase.

This series is similar to (and depends on) the series which scales
msgmni, the number of IPC message queue identifiers, to the amount of
low memory.
While Nadia's previous series scaled msgmni along the memory axis,
hence the message pool (msgmni x msgmnb), this series uses a second
axis: the number of online CPUs.
As well as covering the (cpu,memory) space of machines size, this
reflects the parallelism allowed by lockless send/receive for
in-flight messages in queues (msgmnb / msgmax messages).

The initial scaling is done at initialization of the ipc namespace.
Furthermore, the value becomes dynamic with respect to cpu hotplug,
decreasing/increasing when a cpu is taken offline/online.

The msgmni and msgmnb values become dependent, as the value of msgmni
is computed with respect to the value of msgmnb.

Other solutions could be possible, like using a dbus/hal daemon. This
patches seems light enough not to go to user space. In particular, the
computation formula is simple.

The series is as follows:
. patch 1 introduces the scaling function 
. patch 2 deals with cpu hotplug
. patch 3 finer grain disabling/reenabling scaling mechanism
          (disconnect msgmnb and msgmni)
--- 

The series applies to 2.6.26-rc5-mm3

Compared to the RFC, the following changes have been made:

. reduce use of "scale" word which leads to confusion about the fact
  that this is not directly a performance patch [Nick]
. mention that the formula is simple, not needing logarithm (or user
  space) [Nick]
. example of distribution using patch [Manfred]
. mention hal/dbus daemon [Manfred]
. do not reenable msgmni recomputation when reenabling msgmnb. It
  suffices to do a one shot recomputation [Nadia]
. patch 3 and 4 have been merged with patch 1 [Nadia]
. Integrated documentation with patch [Matt]

Thanks for the help!

. corrected a bug in the last patch
  (forgot to add braces when adding statement in if) 



The last remark of Nadia about the third patch has not been addressed
(other than keeping it like that):
"Doing this, we are completly loosing the benefits of the notification
chains: since the the notifier blocks remain registered + we are
unconditionally adding a test at the top of each recompute routine. But
the other choice would have been to define another notifier chain
dedicated to msgmnb. I'm not convinced about what is the best solution?"

 Documentation/sysctl/kernel.txt |   27 +++++++++++++++++++
 include/linux/ipc_namespace.h   |    4 ++
 include/linux/msg.h             |    5 +++
 ipc/ipc_sysctl.c                |   56 ++++++++++++++++++++++++++++++----------
 ipc/ipcns_notifier.c            |   23 ++++++----------
 ipc/msg.c                       |   23 +++++++++++++---
 ipc/util.c                      |   28 ++++++++++++++++++++
 ipc/util.h                      |    1
 8 files changed, 135 insertions(+), 32 deletions(-)

-- 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists