netdev - cpu utilisation improvement when not doing netif_tx_{stop,wake}

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20080828164638.GA25919@xi.wantstofly.org>
Date:	Thu, 28 Aug 2008 18:46:38 +0200
From:	Lennert Buytenhek <buytenh@...tstofly.org>
To:	davem@...emloft.net, netdev@...r.kernel.org
Cc:	dale@...nsworth.org, nico@....org
Subject: cpu utilisation improvement when not doing netif_tx_{stop,wake}_queue()

I've been hacking on mv643xx_eth to optimise small packet routing
throughput.  The version currently in my tree can route ~190kpps,
versus ~130kpps for the version in 2.6.27.

As part of the hacking, I implemented multiqueue TX, got rid of all
driver-private locking in favor of the per-subqueue _xmit_lock, and
did a bunch of other cleanups.  (Available in the tree below.)

At first, I didn't want to bother with proper synchronisation for
netif_tx_{stop,wake}_queue(), so I just disabled them entirely for the
time being (and would just drop skbs in ->hard_start_xmit() without
warning if the TX ring was full, while returning success).

When I had properly sorted out the locking and turned TX queue flow
control back on again, I started seeing a ~20% CPU usage hit on a TCP
throughput test relative to not doing said flow control.

I.e. transmitting wire speed gigabit over a single TCP session takes
around 90% CPU time (measured with cyclesoak) if I have the TX queue
flow control enabled, but only 70% if I rip the TX queue flow control
out.  (This is on a relatively underpowered ARM box.)

Looking at the profiles doesn't tell me much, and I can't explain what's
going on, but it seems pretty consistent: if I in the tree below revert
to any commit before the "mv643xx_eth: get rid of transmit queue flow
control" one, I get the CPU hit (~90%), whereas with that commit
included it's fine (~70%).

	http://git.marvell.com/?p=mv643xx_eth.git;a=shortlog;h=for-2.6.28
	git://git.marvell.com/mv643xx_eth.git for-2.6.28

Ideas?
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html