linux-kernel - Re: [PATCH] irq: Add node_affinity CPU masks for smarter irqbalance hints

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 24 Nov 2009 20:01:25 +0100
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@...el.com>
CC:	David Miller <davem@...emloft.net>,
	"peterz@...radead.org" <peterz@...radead.org>,
	"arjan@...ux.intel.com" <arjan@...ux.intel.com>,
	"yong.zhang0@...il.com" <yong.zhang0@...il.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"arjan@...ux.jf.intel.com" <arjan@...ux.jf.intel.com>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: [PATCH] irq: Add node_affinity CPU masks for smarter irqbalance
 hints

Peter P Waskiewicz Jr a écrit :

> That's exactly what we're doing in our 10GbE driver right now (isn't
> pushed upstream yet, still finalizing our testing).  We spread to all
> NUMA nodes in a semi-intelligent fashion when allocating our rings and
> buffers.  The last piece is ensuring the interrupts tied to the various
> queues all route to the NUMA nodes those CPUs belong to.  irqbalance
> needs some kind of hint to make sure it does the right thing, which
> today it does not.

sk_buff allocations should be done on the node of the cpu handling rx interrupts.

For rings, I am ok for irqbalance and driver cooperation, in case admin
 doesnt want to change the defaults.

> 
> I don't see how this is complex though.  Driver loads, allocates across
> the NUMA nodes for optimal throughput, then writes CPU masks for the
> NUMA nodes each interrupt belongs to.  irqbalance comes along and looks
> at the new mask "hint," and then balances that interrupt within that
> hinted mask.

So NUMA policy is given by the driver at load time ?

An admin might chose to direct all NIC trafic to a given node, because
its machine has mixed workload. 3 nodes out of 4 for database workload,
one node for network IO...

So if an admin changes smp_affinity, is your driver able to reconfigure itself
and re-allocate all its rings to be on NUMA node chosen by admin ? This is
what I qualify as complex.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/