netdev - Re: [OpenWrt-Devel] [RFC][PATCH 2/2] bgmac: pass received packet to the netif instead of copying it

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CACna6rwi1+2ghyyKonaw=6xVDwipb3=1zUzOJ7dVoNoVyRx9fw@mail.gmail.com>
Date:	Thu, 15 Aug 2013 22:21:56 +0200
From:	Rafał Miłecki <zajec5@...il.com>
To:	Felix Fietkau <nbd@...nwrt.org>
Cc:	OpenWrt Development List <openwrt-devel@...ts.openwrt.org>,
	Network Development <netdev@...r.kernel.org>,
	Hauke Mehrtens <hauke@...ke-m.de>,
	Florian Fainelli <f.fainelli@...il.com>,
	Jonas Gorski <jogo@...nwrt.org>,
	Robert Bradley <robert.bradley1@...il.com>
Subject: Re: [OpenWrt-Devel] [RFC][PATCH 2/2] bgmac: pass received packet to
 the netif instead of copying it

2013/8/15 Felix Fietkau <nbd@...nwrt.org>:
> On 2013-08-15 1:36 PM, Rafał Miłecki wrote:
>> 2013/8/11 Rafał Miłecki <zajec5@...il.com>:
>>> It makes more sense to allocate new (empty) skb and pass it to the
>>> hardware. That way we avoid copying whole packet into new skb which
>>> should result in better performance.
>>
>> I did some testing of this patch using "perf" tool and iperf -s
>> running on the OpenWrt machine (with bgmac supported hardware).
>>
>> So you can see that __copy_user_common usage has really decreased with
>> this patch!
>>
>> Unfortunately it didn't result in better performance... no idea why :(
> Running iperf on the router is not useful as an indicator of routing
> performance. Please focus on tests where you only push traffic through
> the router, not directly to it.

OK, so I started "iperf -s" on notebook plugged into WAN port, and
then played with "iperf -c" on notebook connected to LAN#2.

With some old 3.6.11 based OpenWrt build I got:
[ 4] 0.0-60.0 sec 690 MBytes 96.4 Mbits/sec

With very recent 3.10.4 based OpenWrt build:
[ 4] 0.0-60.0 sec 667 MBytes 93.2 Mbits/sec

After applying my patch on top of that 3.10.4:
[ 5] 0.0-60.0 sec 759 MBytes 106 Mbits/sec

And some dumps from "perf top":

3.10.4
6.75% [kernel] [k] __copy_user_common
6.73% [ip_tables] [k] ipt_do_table
4.33% [kernel] [k] arch_cpu_idle
3.96% [kernel] [k] arch_local_irq_restore
3.42% [bgmac] [k] 0x000007e0
3.35% [nf_conntrack] [k] nf_conntrack_proto_fini
2.72% [nf_conntrack] [k] nf_conntrack_in
2.50% [kernel] [k] __netif_receive_skb_core
2.42% [kernel] [k] r4k_dma_cache_inv
2.38% [kernel] [k] fib_table_lookup
2.20% [kernel] [k] dev_queue_xmit
2.11% [xt_conntrack] [k] 0x00000360
2.10% [kernel] [k] ip_route_input_noref
2.06% [nf_conntrack_ipv4] [k] need_ipv4_conntrack

3.10.4 + 0002-bgmac-pass-received-packet-to-the-netif-instead-of-c.patch
6.09% [ip_tables] [k] ipt_do_table
4.71% [kernel] [k] arch_cpu_idle
4.48% [bgmac] [k] 0x00000d7c
3.50% [nf_conntrack] [k] nf_conntrack_in
3.22% [kernel] [k] arch_local_irq_restore
3.16% [nf_conntrack] [k] nf_conntrack_proto_fini
2.88% [kernel] [k] __netif_receive_skb_core
2.78% [xt_conntrack] [k] 0x0000011c
2.69% [kernel] [k] r4k_dma_cache_inv
2.67% [iptable_nat] [k] 0x000002a0
2.36% [kernel] [k] ip_route_input_noref
2.27% [kernel] [k] ip_rcv
2.25% [nf_conntrack_ipv4] [k] need_ipv4_conntrack
2.23% [kernel] [k] nf_iterate

I've compiled bgmac into the kernel and it seems that the magic 0xd7c
was bgmac_poll.

I'm afraid this "perf top" output doesn't really tell us where to look
for optimizations :| I'll still try Felix ideas tomorrow, but I'm not
sure if they help, since there isn't __copy_user_common anymore in the
"perf top" output...

-- 
Rafał
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html