netdev - Re: [PATCH 1/3] Kernel interfaces for multiqueue aware socket

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1292446118.2603.11.camel@edumazet-laptop>
Date:	Wed, 15 Dec 2010 21:48:38 +0100
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Fenghua Yu <fenghua.yu@...el.com>
Cc:	"David S. Miller" <davem@...emloft.net>,
	John Fastabend <john.r.fastabend@...el.com>,
	Xinan Tang <xinan.tang@...el.com>,
	Junchang Wang <junchangwang@...il.com>,
	netdev <netdev@...r.kernel.org>,
	linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 1/3] Kernel interfaces for multiqueue aware socket

Le mercredi 15 décembre 2010 à 12:02 -0800, Fenghua Yu a écrit :
> From: Fenghua Yu <fenghua.yu@...el.com>
> 
> Multiqueue and multicore provide packet parallel processing methodology.
> Current kernel and network drivers place one queue on one core. But the higher
> level socket doesn't know multiqueue. Current socket only can receive or send
> packets through one network interfaces. In some cases e.g. multi bpf filter
> tcpdump and snort, a lot of contentions come from socket operations like ring
> buffer. Even if the application itself has been fully parallelized and run on
> multi-core systems and NIC handlex tx/rx in multiqueue in parallel, network layer
> and NIC device driver assemble packets to a single, serialized queue. Thus the
> application cannot actually run in parallel in high speed.
> 
> To break the serialized packets assembling bottleneck in kernel, one way is to
> allow socket to know multiqueue associated with a NIC interface. So each socket
> can handle tx/rx in one queue in parallel.
> 
> Kernel provides several interfaces by which sockets can be bound to rx/tx queues.
> User applications can configure socket by providing several sockets that each
> bound to a single queue, applications can get data from kernel in parallel. After
> that, competitions mentioned above can be removed.
> 
> With this patch, the user-space receiving speed on a Intel SR1690 server with
> a single L5640 6-core processor and a single ixgbe-based NIC goes from 0.73Mpps
> to 4.20Mpps, nearly a linear speedup. A Intel SR1625 server two E5530 4-core
> processors and a single ixgbe-based NIC goes from 0.80Mpps to 4.6Mpps. We noticed
> the performance penalty comes from NUMA memory allocation.
> 

??? please elaborate on these NUMA memory allocations. This should be OK
after commit 564824b0c52c34692d (net: allocate skbs on local node)

> This patch set provides kernel ioctl interfaces for user space. User space can
> either directly call the interfaces or libpcap interfaces can be further provided
> on the top of the kernel ioctl interfaces.

So, say we have 8 queues, you want libpcap opens 8 sockets, and bind
them to each queue. Add a bpf filter to each one of them. This seems not
generic way, because it wont work for an UDP socket for example.
And you already can do this using SKF_AD_QUEUE (added in commit
d19742fb)

Also your AF_PACKET patch only address mmaped sockets.



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html