netdev - Re: [PATCH net-next-2.6] sched: use xps information for qdisc NUMA affinity

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 30 Nov 2010 20:07:27 +0100
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	David Miller <davem@...emloft.net>
Cc:	therbert@...gle.com, netdev@...r.kernel.org,
	bhutchings@...arflare.com
Subject: Re: [PATCH net-next-2.6] sched: use xps information for qdisc NUMA
 affinity

Le mardi 30 novembre 2010 à 10:48 -0800, David Miller a écrit :

> Most drivers do, and all drivers ought to, allocate DMA queues and
> whatnot when the interface is brought up.
> 
> That solves this particular issue.
> 
> For example, drivers/net/niu.c does this by calling
> niu_alloc_channels() via niu_open().
> 
> The only thing we really can't handle currently is the netdev
> itself (and the associated driver private).  Jesse Brandeburg
> has been reminding me about this over and over :-)
> 
> There might be some things we can even do about that part.  For
> example, we can put all of the things the driver touches in the
> RX and TX fast paths via indirect pointers and therefore be able
> to allocate and reallocate those portions as we want long after
> device registry.
> 
> Doing the core netdev struct itself is too hard because it sits
> in so many tables.

netdev struct itself is shared by all cpus, so there is no real choice,
unless you know one netdev will be used by a restricted set of
cpus/nodes... Probably very unlikely in practice.

This can probably be done right now with 
numactl .... modprobe ...

We could change (only on NUMA setups maybe)

struct netdev_queue *_tx;

to a

struct netdev_queue **_tx;

and allocate each "struct netdev_queue" on appropriate node, but adding
one indirection level might be overkill...

For very hot small structures, (one or two cache lines), I am not sure
its worth the pain.

Ben, could you remind us what was your ethtool interface ?

Something to setup a NUMA map for RX queues, TX queues ?

I can probably play with bnx2x and custom module param to test how it
can help raw performance...

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html