netdev - Multiqueue pktgen and ingress path (Was: [PATCH v5 2/2] pktgen: introduce xmit_mode '<start_xmit|netif

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150508174927.5b1ecdd1@redhat.com>
Date:	Fri, 8 May 2015 17:49:27 +0200
From:	Jesper Dangaard Brouer <brouer@...hat.com>
To:	Daniel Borkmann <daniel@...earbox.net>
Cc:	Alexei Starovoitov <ast@...mgrid.com>, netdev@...r.kernel.org,
	Eric Dumazet <eric.dumazet@...il.com>, brouer@...hat.com
Subject: Multiqueue pktgen and ingress path (Was: [PATCH v5 2/2] pktgen:
 introduce xmit_mode '<start_xmit|netif_receive>')

On Fri, 8 May 2015 17:39:00 +0200
Jesper Dangaard Brouer <brouer@...hat.com> wrote:

> On Thu, 07 May 2015 19:11:58 +0200 Daniel Borkmann <daniel@...earbox.net> wrote:
> 
> > On 05/07/2015 06:28 PM, Alexei Starovoitov wrote:  
> > > On 5/7/15 7:35 AM, Jesper Dangaard Brouer wrote:  
> > >> From: Alexei Starovoitov <ast@...mgrid.com>
> > >>  
> [...snip...]
> 
> > > btw, I've started to work on a patch on top of this one that allows
> > > multiple pktgen threads to submit into the same netdev.
> > > I've used it to stress test removal of spin_lock in ingress qdisc.
> > > The idea is to add another 'name' parameter to command:
> > > 'add_device name dev'
> > > 'name' will be used to identify this pktgen thread in /proc
> > > and 'dev' used as target net_device.
> > > I think it will be useful for start_xmit testing as well.
> > > I wonder why it wasn't done earlier? The queue configuration is
> > > already supported.  
> > 
> > You mean other than below commit (iow independant of queue mapping)?
> > 
> > commit e6fce5b916cd7f7f79b2b3e53ba74bbfc1d7cf8b
> > Author: Robert Olsson <robert.olsson@....uu.se>
> > Date:   Thu Aug 7 02:23:01 2008 -0700
> > 
> >      pktgen: multiqueue etc.  
> 
> For completeness and others reading this threads...
> 
> Pktgen multiqueue is already supported via mentioned commit, which adds
> the device naming scheme: "add_device dev@...ber"
> 
> And yes, the documentation does not seem to mention this.  I've been
> using it for years now... My scripts[1] take param "-t" for "threads".
> [1] https://github.com/netoptimizer/network-testing/tree/master/pktgen
> 
> I've added a more plain version of a script, based on yours, below my
> signature.

Now attached.

> The funny thing now is that scaling does not "happen" as we stall on:
>    atomic_long_inc(&skb->dev->rx_dropped);

More interesting observations with the mentioned script (now attached).

On my system the scaling stopped a 24Mpps, when I increased the number
of threads the collective scaling was stuck at 24Mpps.

Then I simply removed/compiled-out the:
 atomic_long_inc(&skb->dev->rx_dropped);

And after that change, the scaling is basically infinite/perfect.

Single thread performance increased from 24.7Mpps to 31.1Mpps, which
corresponds perfectly with the cost of an atomic operation on this HW
(8.25ns).

Diff to before:
 * (1/24700988*10^9)-(1/31170819*10^9) = 8.40292328196 ns

When increasing the threads now, they all basically run at 31Mpps.
Tried it upto 12 threads.


I'm quite puzzled why a single atomic op could "freeze" my system from
scaling beyond 24Mpps.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Download attachment "pktgen_multiqueue01_xmit_mode.sh" of type "application/x-shellscript" (1217 bytes)