| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1285250692.1837.98.camel@holzheu-laptop>
Date: Thu, 23 Sep 2010 16:04:52 +0200
From: Michael Holzheu <holzheu@...ux.vnet.ibm.com>
To: Shailabh Nagar <nagar1234@...ibm.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Venkatesh Pallipadi <venki@...gle.com>,
Peter Zijlstra <a.p.zijlstra@...llo.nl>,
Suresh Siddha <suresh.b.siddha@...el.com>,
John stultz <johnstul@...ibm.com>,
Thomas Gleixner <tglx@...utronix.de>,
Oleg Nesterov <oleg@...hat.com>,
Balbir Singh <balbir@...ux.vnet.ibm.com>,
Ingo Molnar <mingo@...e.hu>,
Heiko Carstens <heiko.carstens@...ibm.com>,
Martin Schwidefsky <schwidefsky@...ibm.com>
Cc: linux-s390@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [RFC][PATCH 10/10] taststats: User space with ptop tool
***********************
Warning: Your file, s390-tools-taskstats.tar.bz2, contains more than 32 files after decompression and cannot be scanned.
***********************
Taskstats user space
The attached tarball "s390-tools-taskstasts.tar.bz2" contains user space
code that exploits the taskstasts-top kernel patches. This is early
code and probably still a lot of work has to be done here. The
code should build and work on all architectures, not only on s390.
libtaskstats user space library
-------------------------------
include/libtaskstats.h API definition
libtaskstats_nl API implementation based on libnl 1.1
libtaskstats_proc Partial API implementation using
new /proc/taskstats
libtaskstats snapshot user space library
----------------------------------------
include/libtaskstats_snap.h API definition
libtaskstats_snap/snap_netlink.c API implementation based on
libtaskstats
Snapshot library test program
-----------------------------
ts_snap_test/ts_snap_test.c Simple program that uses snapshot library
Precise top user space program (ptop)
-------------------------------------
ptop/dg_libtaskstats.c Data gatherer using taskstats interface
To enable steal time calculation for non s390
modify l_calc_sttime_old() and replact "#if 0"
with "#if 1".
ptop/sd_core.c Code for ctime accounting
HOWTO build:
============
1.Install libnl-1.1-5 and libnl-1.1-5-devel
If this is not possible, you can still build the proc/taskstats based
code:
* Remove libtaskstats_nl from the top level Makefile
* Remove ptop_old_nl, ptop_new_nl and ptop_snap_nl from the
"ptop" Makefile
2.Build s390-tools:
# tar xfv s390-tools.tar.bz2
# cd s390-tools
# make
HOWTO use ptop:
===============
In the ptop sub directory there are built five versions of ptop:
* ptop_old_nl: ptop using the old TASKSTATS_CMD_ATTR_PID netlink
command together with reading procfs to find
running tasks
* ptop_new_nl: ptop using the new TASKSTATS_CMD_ATTR_PIDS
netlink command.
This tool only shows tasks that consumed CPU time
in the last interval.
* ptop_new_proc: ptop using the new TASKSTATS_CMD_ATTR_PIDS ioctl on
/proc/taskstats.
This tool only shows tasks that consumed CPU time
in the last interval.
* ptop_snap_nl: ptop using the snapshot library with underlying
netlink taskstats library
* ptop_snap_proc: ptop using the snapshot library with underlying
taskstats library that uses /proc/taskstats
First results (on s390):
========================
TEST1: System with many sleeping tasks
--------------------------------------
for ((i=0; i < 1000; i++))
do
sleep 1000000 &
done
VVVV
pid user sys ste total Name
(#) (%) (%) (%) (%) (str)
541 0.37 2.39 0.10 2.87 top
3645 2.13 1.12 0.14 3.39 ptop_old_nl
3591 2.20 0.59 0.12 2.92 ptop_snap_nl
3694 2.16 0.26 0.10 2.51 ptop_snap_proc
3792 0.03 0.06 0.00 0.09 ptop_new_nl
3743 0.03 0.05 0.00 0.07 ptop_new_proc
^^^^
The ptop user space code is not optimized for a large amount of tasks,
therefore we should concentrate on the system (sys) time. Update time is
2 seconds for all top programs.
* Old top command:
Because top has to read about 1000 procfs directories, system time
is very high (2.39%).
* ptop_new_xxx:
Because only active tasks are transferred, the CPU consumption is very
low (0.05-0.06% system time).
* ptop_snap_nl/ptop_old_nl:
The new netlink TASKSTATS_CMD_ATTR_PIDS command only consumes about
50% of the CPU time (0.59%) compared to the usage of multiple
TASKSTATS_CMD_ATTR_PID commands (ptop_old_nl / 1.12%) and scanning
procfs to find out running tasks.
* ptop_snap_proc/ptop_snap_nl:
Using the proc/taskstats interface (0.26%) consumes much less system
time than the netlink interface (0.59%).
TEST2: Show snapshot consistency with system that is 100% busy
--------------------------------------------------------------
System with 3 CPUs:
for ((i=0; i < $(cat /proc/cpuinfo | grep "^processor" | wc -l); i
++))
do
./loop &
done
cd linux-2.6.35
make -j 5
# ptop_snap_proc
VVVVV
pid user sys ste cuser csys cste total Elap+ Name
(#) (%) (%) (%) (%) (%) (%) (%) (hm) (str)
8374 75.48 0.41 1.34 0.00 0.00 0.00 77.24 0:01 loop
8377 73.97 0.27 1.06 0.00 0.00 0.00 75.31 0:01 loop
8371 70.61 0.38 1.38 0.00 0.00 0.00 72.38 0:01 loop
10093 0.17 0.30 0.00 25.90 38.19 0.52 65.07 0:00 make
10548 0.15 0.12 0.00 1.75 4.21 0.06 6.29 0:00 make
...
V:V:S 220.84 2.84 3.86 28.14 43.71 0.60 300.00 0:16
^^^^^^
With the snapshot mechanism the sum of all tasks CPU times (user +
system + steal + cuser + csystem + csteal) will be exactly 300.00% CPU
time with this testcase. Using ptop_snap_proc this works fine on
s390. Unfortunately on x86 the numbers are not as good as on s390.
Download attachment "s390-tools-taskstats.tar.bz2" of type "application/x-bzip-compressed-tar" (43162 bytes)
Powered by blists - more mailing lists