| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1289495493.1902.23.camel@holzheu-laptop>
Date: Thu, 11 Nov 2010 18:11:33 +0100
From: Michael Holzheu <holzheu@...ux.vnet.ibm.com>
To: Shailabh Nagar <nagar1234@...ibm.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Venkatesh Pallipadi <venki@...gle.com>,
Suresh Siddha <suresh.b.siddha@...el.com>,
Peter Zijlstra <a.p.zijlstra@...llo.nl>,
Ingo Molnar <mingo@...e.hu>, Oleg Nesterov <oleg@...hat.com>,
John stultz <johnstul@...ibm.com>,
Thomas Gleixner <tglx@...utronix.de>,
Balbir Singh <balbir@...ux.vnet.ibm.com>,
Martin Schwidefsky <schwidefsky@...ibm.com>,
Heiko Carstens <heiko.carstens@...ibm.com>,
Roland McGrath <roland@...hat.com>
Cc: linux-kernel@...r.kernel.org, linux-s390@...r.kernel.org
Subject: [RFC][PATCH v2 7/7] taskstats: Precise process accounting user
space
Taskstats user space
The attached tarball "s390-tools-taskstats.tar.bz2" contains user space code
that exploits the taskstasts-top kernel patches. This is early code and
probably still a lot of work has to be done here. The code should build
and work on all architectures, not only on s390.
libtaskstats user space library
-------------------------------
include/libtaskstats.h API definition
libtaskstats_nl API implementation based on libnl 1.1
libtaskstats_proc Partial API implementation using new /proc/taskstats
libtaskstats snapshot user space library
----------------------------------------
include/libtaskstats_snap.h API definition
libtaskstats_snap/snap_netlink.c API implementation based on libtaskstats
Snapshot library test program
-----------------------------
ts_snap_test/ts_snap_test.c Simple program that uses snapshot library
Precise top user space program (ptop)
-------------------------------------
ptop/dg_libtaskstats.c Data gatherer using taskstats interface
To disable steal time calculation for non s390
modify l_calc_sttime_old() and replace "#if 1"
with "#if 0".
ptop/sd_core.c Code for ctime accounting
HOWTO build:
============
1.Install libnl-1.1-5 and libnl-1.1-5-devel
If this is not possible, you can still build the proc/taskstats based code:
* Remove libtaskstats_nl from the top level Makefile
* Remove ptop_old_nl, ptop_new_nl and ptop_snap_nl from the "ptop" Makefile
2.Build s390-tools:
# tar xfv s390-tools.tar.bz2
# cd s390-tools
# make
HOWTO use ptop:
===============
In the ptop sub directory there are built five versions of ptop:
* ptop_old_nl: ptop using the old TASKSTATS_CMD_ATTR_PID netlink command
together with reading procfs to find running tasks
* ptop_new_nl: ptop using the new TASKSTATS_CMD_ATTR_PIDS netlink command.
This tool only shows tasks that consumed CPU time in the
last interval.
* ptop_new_proc: ptop using the new TASKSTATS_CMD_ATTR_PIDS ioctl on
/proc/taskstats.
This tool only shows tasks that consumed CPU time in the
last interval.
* ptop_snap_nl: ptop using the snapshot library with underlying netlink
taskstats library
* ptop_snap_proc: ptop using the snapshot library with underlying taskstats
library that uses /proc/taskstats
First results (on s390):
========================
TEST1: System with many sleeping tasks
--------------------------------------
for ((i=0; i < 1000; i++))
do
sleep 1000000 &
done
# ptop_new_proc
VVVV
pid user sys ste total Name
(#) (%) (%) (%) (%) (str)
541 0.37 2.39 0.10 2.87 top
3645 2.13 1.12 0.14 3.39 ptop_old_nl
3591 2.20 0.59 0.12 2.92 ptop_snap_nl
3694 2.16 0.26 0.10 2.51 ptop_snap_proc
3792 0.03 0.06 0.00 0.09 ptop_new_nl
3743 0.03 0.05 0.00 0.07 ptop_new_proc
^^^^
The ptop user space code is not optimized for a large amount of tasks,
therefore we should concentrate on the system (sys) time. Update time is
2 seconds for all top programs.
* Old top command:
Because top has to read about 1000 procfs directories, system time is very
high (2.39%).
* ptop_new_xxx:
Because only active tasks are transferred, the CPU consumption is very low
(0.05-0.06% system time).
* ptop_snap_nl/ptop_old_nl:
The new netlink TASKSTATS_CMD_ATTR_PIDS command only consumes about 50% of
the CPU time (0.59%) compared to the usage of multiple TASKSTATS_CMD_ATTR_PID
commands (ptop_old_nl / 1.12%) and scanning procfs to find out running tasks.
* ptop_snap_proc/ptop_snap_nl:
Using the proc/taskstats interface (0.26%) consumes much less system time
than the netlink interface (0.59%).
TEST2: Show snapshot consistency with system that is 100% busy
--------------------------------------------------------------
System with 2 CPUs:
for ((i=0; i < $(cat /proc/cpuinfo | grep "^processor" | wc -l); i++))
do
./loop &
done
cd linux-2.6
make -j 5
# ptop_snap_proc
pid user sys stea cuser csys cstea xuser xsys xstea total Name
(#) (%) (%) (%) (%) (%) (%) (%) (%) (%) (%) (str)
2802 43.22 0.35 0.22 0.00 0.00 0.00 0.00 0.00 0.00 43.79 loop
2799 35.96 0.33 0.21 0.00 0.00 0.00 0.00 0.00 0.00 36.50 loop
2811 0.04 0.05 0.00 23.22 12.97 0.19 0.00 0.00 0.00 36.46 make
2796 35.80 0.32 0.19 0.00 0.00 0.00 0.00 0.00 0.00 36.30 loop
2987 15.93 2.14 0.07 8.23 3.12 0.06 0.00 0.00 0.00 29.53 make
3044 11.56 1.72 0.22 0.00 0.00 0.00 0.00 0.00 0.00 13.50 make
3053 1.92 0.73 0.01 0.00 0.00 0.00 0.00 0.00 0.00 2.65 make
....
V:V:S 144.76 6.24 1.22 31.44 16.09 0.25 0.00 0.00 0.00 200.00
^^^^^^
With the snapshot mechanism the sum of all tasks CPU times will be exactly
200.00% CPU time with this testcase. The following CPU times are used:
* user/sys/stea: Time that has been consumed by task itself
* cuser/csys/cstea: All time that has been consumed by dead children of
process.
* xuser/xsys/xstea: Time that has been consumed by dead threads of thread
group of task
Note that the CPU times on x86 are not as precise as on s390.
Download attachment "s390-tools-taskstats.tar.bz2" of type "application/x-bzip-compressed-tar" (42754 bytes)
Powered by blists - more mailing lists