linux-kernel - RFC: NUMA modifications to cyclictest

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100119171439.24726f2e@torg>
Date:	Tue, 19 Jan 2010 17:14:39 -0600
From:	Clark Williams <williams@...hat.com>
To:	RT <linux-rt-users@...r.kernel.org>
Cc:	LKML <linux-kernel@...r.kernel.org>, Carsten Emde <ce@...g.ch>,
	John Kacur <jkacur@...hat.com>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: RFC: NUMA modifications to cyclictest

RT-ers,

Lately we've been struggling with some performance issues on high-core
count (>16 cores) NUMA machines with the RT kernel. During the course
of troubleshooting this issue, we tried using the 'numactl' program to
constrain our measurement testing tool (rteval) to a particular memory
node, rather than letting everything float. Doing so showed marked
improvement in both max latency and jitter.  While this doesn't solve
our performance problems I thought it might make sense to have a --numa
mode for cylictest that compliments the --smp mode just added. 

The big difference here is that when using --numa, each measurement
thread (one per cpu) has it's stack allocated from the memory node
associated with it's cpu. Also, the major data structures for each
thread (parameter block, statistics block and histogram) are allocated
from the appropriate node. This is done with calls into libnuma,
which means this will add a dependency on libnuma. 

The intent is to measure latency on a numa system in the same way a
well-written RT application would run on a NUMA machine, that is
minimizing the off-node memory references. 

If you're interested in looking at this, please pull the numa branch
from my git repo at:

	git://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rt-tests.git

and let me know if you find bugs or disagree with the approach. 

Thanks,
Clark

Download attachment "signature.asc" of type "application/pgp-signature" (199 bytes)