netdev - Re: [PATCH net-next] pktgen: fix packet generation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <55522113.8050708@plumgrid.com>
Date:	Tue, 12 May 2015 08:49:39 -0700
From:	Alexei Starovoitov <ast@...mgrid.com>
To:	Jesper Dangaard Brouer <brouer@...hat.com>
CC:	"David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>,
	Daniel Borkmann <daniel@...earbox.net>, netdev@...r.kernel.org
Subject: Re: [PATCH net-next] pktgen: fix packet generation

On 5/12/15 1:19 AM, Jesper Dangaard Brouer wrote:
> On Mon, 11 May 2015 15:19:48 -0700
> Alexei Starovoitov <ast@...mgrid.com> wrote:
>
>> pkt_gen->last_ok was not set properly, so after the first burst
>> pktgen instead of allocating new packet, will reuse old one, advance
>> eth_type_trans further, which would mean the stack will be seeing very
>> short bogus packets.
>>
>> Fixes: 62f64aed622b ("pktgen: introduce xmit_mode '<start_xmit|netif_receive>'")
>> Signed-off-by: Alexei Starovoitov <ast@...mgrid.com>
>> ---
>> This bug slipped through due to all code refactoring and can be seen
>> after clean reboot. If taps, rps or tx mode was used at least once,
>> the bug will be hidden.
>>
>> Note to users: if you don't see ip_rcv() in your perf profile, it means
>> you were hitting this.
>> As commit log of 62f64aed622b is saying, the baseline perf profile
>> should look like:
>>     37.69%  kpktgend_0   [kernel.vmlinux]  [k] __netif_receive_skb_core
>>     25.81%  kpktgend_0   [kernel.vmlinux]  [k] kfree_skb
>>      7.22%  kpktgend_0   [kernel.vmlinux]  [k] ip_rcv
>>      5.68%  kpktgend_0   [pktgen]          [k] pktgen_thread_worker
>>
>> Jesper, that explains why you were seeing hot:
>> atomic_long_inc(&skb->dev->rx_dropped);
>
> Acked-by: Jesper Dangaard Brouer <brouer@...hat.com>

thanks!

> Yes, just confirmed that this problem is gone. E.g. the multiqueue
> script now scales without hitting this "skb->dev->rx_dropped".

great.

> Good this got fixed, as my plan is to use this to profile the memory
> allocators fast path for SKB alloc/free.
>
> Setting "burst = 0" (and flag NO_TIMESTAMP):
>   Device: eth4@0
>    3938513pps 1890Mb/sec (1890486240bps) errors: 10000000
>
> Thus, performance hit from 22.1Mpps to 3.9Mpps, thus 209 nanosec more
> expensive.  20% is the cost of pktgen itself, still I'm surprised that
> the hit is this big, as this should hit the most optimal cache-hot case
> of SKB alloc/free.

I tried similar and I think I've seen ip_send_check(iph); called by
pktgen->fill_packet_ipv4 to be quite hot, so burst=1 will be measuring
quite a bit more than just skb alloc/free. I think your skb allocator
micro bench was more accurate.

btw, this multi-core pktgen into netif_receive skb exposed all
spin_locks in tc actions. We need to convert them to rcu.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html