netdev - Re: [net-next 02/11] igb: Use node specific allocations for the q

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 10 Oct 2011 10:02:07 -0700
From:	Alexander Duyck <alexander.h.duyck@...el.com>
To:	Andi Kleen <andi@...stfloor.org>
CC:	Jeff Kirsher <jeffrey.t.kirsher@...el.com>, davem@...emloft.net,
	netdev@...r.kernel.org, gospo@...hat.com, sassmann@...hat.com
Subject: Re: [net-next 02/11] igb: Use node specific allocations for the q_vectors
 and rings

On 10/10/2011 09:32 AM, Andi Kleen wrote:
>> The RR configuration is somewhat arbitrary.  However it is still better
>> than dumping everyting on a single node, and it works with the
>> configuration when the rings numbers line up with the CPU numbers since
>> normally the CPUs are RR on the nodes.  From what I have seen it does
>> work quite well and it prevents almost all cross-node memory accesses
>> when running a routing workload.
> 
> Ok so it's optimized for one specific workload. I'm sure you'll
> find some other workload where it doesn't work out.

It isn't that I optimized it for one specific workload.  I was just
citing that specific workload as one of the ones seeing the advantage.

> I suppose it's hard to get right in the general case, but best
> would be if ethtool had a nice and easy interface to set it at least.

The general case is never right for this it seems like.  At least in
this case it becomes much easier to line up the memory and interrupts so
that they are all affinitized to the same core.  From there RPS/RFS can
typically be used to spread out the work more if necessary.

> However one disadvantage of that patch over the existing state of the
> art (numactl modprobe ...) is that there's no way to override the placement
> now. So if you do the forced RR I think you need the ethtool part too,
> or at least some parameter to turn it off.
> 
> -Andi

The counter argument to that though is that the approach you mention
always limits you to one node.  At least with this approach we are
spread out over multiple nodes so that we can make full use of the
memory bandwidth on the system.

Thanks,

Alex
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html