netdev - Re: [OpenWrt-Devel] [RFC][PATCH 2/2] bgmac: pass received packet to the netif instead of copying it

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 19 Aug 2013 06:36:27 +0200
From:	Felix Fietkau <nbd@...nwrt.org>
To:	Rafał Miłecki <zajec5@...il.com>
CC:	OpenWrt Development List <openwrt-devel@...ts.openwrt.org>,
	Network Development <netdev@...r.kernel.org>,
	Hauke Mehrtens <hauke@...ke-m.de>,
	Florian Fainelli <f.fainelli@...il.com>,
	Jonas Gorski <jogo@...nwrt.org>,
	Robert Bradley <robert.bradley1@...il.com>
Subject: Re: [OpenWrt-Devel] [RFC][PATCH 2/2] bgmac: pass received packet
 to the netif instead of copying it

On 2013-08-15 10:21 PM, Rafał Miłecki wrote:
> 2013/8/15 Felix Fietkau <nbd@...nwrt.org>:
>> On 2013-08-15 1:36 PM, Rafał Miłecki wrote:
>>> 2013/8/11 Rafał Miłecki <zajec5@...il.com>:
>>>> It makes more sense to allocate new (empty) skb and pass it to the
>>>> hardware. That way we avoid copying whole packet into new skb which
>>>> should result in better performance.
>>>
>>> I did some testing of this patch using "perf" tool and iperf -s
>>> running on the OpenWrt machine (with bgmac supported hardware).
>>>
>>> So you can see that __copy_user_common usage has really decreased with
>>> this patch!
>>>
>>> Unfortunately it didn't result in better performance... no idea why :(
>> Running iperf on the router is not useful as an indicator of routing
>> performance. Please focus on tests where you only push traffic through
>> the router, not directly to it.
> 
> OK, so I started "iperf -s" on notebook plugged into WAN port, and
> then played with "iperf -c" on notebook connected to LAN#2.
> 
> With some old 3.6.11 based OpenWrt build I got:
> [ 4] 0.0-60.0 sec 690 MBytes 96.4 Mbits/sec
> 
> With very recent 3.10.4 based OpenWrt build:
> [ 4] 0.0-60.0 sec 667 MBytes 93.2 Mbits/sec
> 
> After applying my patch on top of that 3.10.4:
> [ 5] 0.0-60.0 sec 759 MBytes 106 Mbits/sec
> 
> And some dumps from "perf top":
> 
> 3.10.4
> 6.75% [kernel] [k] __copy_user_common
> 6.73% [ip_tables] [k] ipt_do_table
> 4.33% [kernel] [k] arch_cpu_idle
> 3.96% [kernel] [k] arch_local_irq_restore
> 3.42% [bgmac] [k] 0x000007e0
> 3.35% [nf_conntrack] [k] nf_conntrack_proto_fini
> 2.72% [nf_conntrack] [k] nf_conntrack_in
> 2.50% [kernel] [k] __netif_receive_skb_core
> 2.42% [kernel] [k] r4k_dma_cache_inv
> 2.38% [kernel] [k] fib_table_lookup
> 2.20% [kernel] [k] dev_queue_xmit
> 2.11% [xt_conntrack] [k] 0x00000360
> 2.10% [kernel] [k] ip_route_input_noref
> 2.06% [nf_conntrack_ipv4] [k] need_ipv4_conntrack
> 
> 3.10.4 + 0002-bgmac-pass-received-packet-to-the-netif-instead-of-c.patch
> 6.09% [ip_tables] [k] ipt_do_table
> 4.71% [kernel] [k] arch_cpu_idle
> 4.48% [bgmac] [k] 0x00000d7c
> 3.50% [nf_conntrack] [k] nf_conntrack_in
> 3.22% [kernel] [k] arch_local_irq_restore
> 3.16% [nf_conntrack] [k] nf_conntrack_proto_fini
> 2.88% [kernel] [k] __netif_receive_skb_core
> 2.78% [xt_conntrack] [k] 0x0000011c
> 2.69% [kernel] [k] r4k_dma_cache_inv
> 2.67% [iptable_nat] [k] 0x000002a0
> 2.36% [kernel] [k] ip_route_input_noref
> 2.27% [kernel] [k] ip_rcv
> 2.25% [nf_conntrack_ipv4] [k] need_ipv4_conntrack
> 2.23% [kernel] [k] nf_iterate
> 
> I've compiled bgmac into the kernel and it seems that the magic 0xd7c
> was bgmac_poll.
> 
> I'm afraid this "perf top" output doesn't really tell us where to look
> for optimizations :| I'll still try Felix ideas tomorrow, but I'm not
> sure if they help, since there isn't __copy_user_common anymore in the
> "perf top" output...
What's the CPU load while passing traffic without running perf?
Have you tested bridging performance?

- Felix
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html