linux-kernel - Re: hackbench regression due to commit 9dfc6e68bfe6e

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1270632033.2091.875.camel@edumazet-laptop>
Date:	Wed, 07 Apr 2010 11:20:33 +0200
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	"Zhang, Yanmin" <yanmin_zhang@...ux.intel.com>
Cc:	Christoph Lameter <cl@...ux-foundation.org>,
	netdev <netdev@...r.kernel.org>, Tejun Heo <tj@...nel.org>,
	Pekka Enberg <penberg@...helsinki.fi>, alex.shi@...el.com,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"Ma, Ling" <ling.ma@...el.com>,
	"Chen, Tim C" <tim.c.chen@...el.com>,
	Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: hackbench regression due to commit 9dfc6e68bfe6e

Le mercredi 07 avril 2010 à 17:07 +0800, Zhang, Yanmin a écrit :
> > 
> > One experiment on your Nehalem machine would be to change hackbench so
> > that each group (20 senders/ 20 receivers) run on a particular NUMA
> > node.
> I expect process scheduler to work well in scheduling different groups
> to different nodes.
> 
> I suspected dynamic percpu data didn't take care of NUMA, but kernel dump shows
> it does take care of NUMA.
> 

hackbench allocates all unix sockets on one single node, then
forks/spans its children.

Thats huge node imbalance.

You can see this with lsof on a running hackbench :


# lsof -p 14802
COMMAND     PID USER   FD   TYPE             DEVICE    SIZE     NODE NAME
hackbench 14802 root  cwd    DIR              104,7    4096 12927240 /data/src/linux-2.6
hackbench 14802 root  rtd    DIR              104,2    4096        2 /
hackbench 14802 root  txt    REG              104,2   17524   697317 /usr/bin/hackbench
hackbench 14802 root  mem    REG              104,2  112212   558042 /lib/ld-2.3.4.so
hackbench 14802 root  mem    REG              104,2 1547588   558043 /lib/tls/libc-2.3.4.so
hackbench 14802 root  mem    REG              104,2  107928   557058 /lib/tls/libpthread-2.3.4.so
hackbench 14802 root  mem    REG                0,0                0 [heap] (stat: No such file or directory)
hackbench 14802 root    0u   CHR              136,0                3 /dev/pts/0
hackbench 14802 root    1u   CHR              136,0                3 /dev/pts/0
hackbench 14802 root    2u   CHR              136,0                3 /dev/pts/0
hackbench 14802 root    3u  unix 0xffff8800ac0da100            28939 socket
hackbench 14802 root    4u  unix 0xffff8800ac0da400            28940 socket
hackbench 14802 root    5u  unix 0xffff8800ac0da700            28941 socket
hackbench 14802 root    6u  unix 0xffff8800ac0daa00            28942 socket
hackbench 14802 root    8u  unix 0xffff8800aeac1800            28984 socket
hackbench 14802 root    9u  unix 0xffff8800aeac1e00            28986 socket
hackbench 14802 root   10u  unix 0xffff8800aeac2400            28988 socket
hackbench 14802 root   11u  unix 0xffff8800aeac2a00            28990 socket
hackbench 14802 root   12u  unix 0xffff8800aeac3000            28992 socket
hackbench 14802 root   13u  unix 0xffff8800aeac3600            28994 socket
hackbench 14802 root   14u  unix 0xffff8800aeac3c00            28996 socket
hackbench 14802 root   15u  unix 0xffff8800aeac4200            28998 socket
hackbench 14802 root   16u  unix 0xffff8800aeac4800            29000 socket
hackbench 14802 root   17u  unix 0xffff8800aeac4e00            29002 socket
hackbench 14802 root   18u  unix 0xffff8800aeac5400            29004 socket
hackbench 14802 root   19u  unix 0xffff8800aeac5a00            29006 socket
hackbench 14802 root   20u  unix 0xffff8800aeac6000            29008 socket
hackbench 14802 root   21u  unix 0xffff8800aeac6600            29010 socket
hackbench 14802 root   22u  unix 0xffff8800aeac6c00            29012 socket
hackbench 14802 root   23u  unix 0xffff8800aeac7200            29014 socket
hackbench 14802 root   24u  unix 0xffff8800aeac0f00            29016 socket
hackbench 14802 root   25u  unix 0xffff8800aeac0900            29018 socket
hackbench 14802 root   26u  unix 0xffff8800aeac7b00            29020 socket
hackbench 14802 root   27u  unix 0xffff8800aeac7500            29022 socket

All sockets structures (where all _hot_ locks reside) are on a single node.



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/