netdev - [Just for fun] loopback: avoid softirq on most transmits

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 28 Oct 2010 21:48:46 +0200
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	David Miller <davem@...emloft.net>
Cc:	netdev <netdev@...r.kernel.org>
Subject: [Just for fun] loopback: avoid softirq on most transmits

With the introduction of xmit_recursion percpu variable, its pretty
cheap to check our recursion level in loopback transmit, and avoid
raising softirq.

Signed-off-by: Eric Dumazet <eric.dumazet@...il.com>
---
tbench faster by 4%, sorry I couldnt resist...

 drivers/net/loopback.c    |   13 +++++++++++--
 include/linux/netdevice.h |    3 +++
 net/core/dev.c            |    2 +-
 3 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/net/loopback.c b/drivers/net/loopback.c
index 2d9663a..5bd73c0 100644
--- a/drivers/net/loopback.c
+++ b/drivers/net/loopback.c
@@ -74,7 +74,7 @@ static netdev_tx_t loopback_xmit(struct sk_buff *skb,
 				 struct net_device *dev)
 {
 	struct pcpu_lstats *lb_stats;
-	int len;
+	int len, res;
 
 	skb_orphan(skb);
 
@@ -84,7 +84,16 @@ static netdev_tx_t loopback_xmit(struct sk_buff *skb,
 	lb_stats = this_cpu_ptr(dev->lstats);
 
 	len = skb->len;
-	if (likely(netif_rx(skb) == NET_RX_SUCCESS)) {
+
+	/*
+	 * avoid raising softirq if our recursion level is low
+	 */
+	if (likely(__this_cpu_read(xmit_recursion) <= 2))
+		res = netif_receive_skb(skb);
+	else
+		res = netif_rx(skb);
+
+	if (likely(res == NET_RX_SUCCESS)) {
 		u64_stats_update_begin(&lb_stats->syncp);
 		lb_stats->bytes += len;
 		lb_stats->packets++;
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 072652d..918330b 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1741,6 +1741,9 @@ extern void dev_kfree_skb_any(struct sk_buff *skb);
 extern int		netif_rx(struct sk_buff *skb);
 extern int		netif_rx_ni(struct sk_buff *skb);
 #define HAVE_NETIF_RECEIVE_SKB 1
+
+DECLARE_PER_CPU(int, xmit_recursion);
+
 extern int		netif_receive_skb(struct sk_buff *skb);
 extern gro_result_t	dev_gro_receive(struct napi_struct *napi,
 					struct sk_buff *skb);
diff --git a/net/core/dev.c b/net/core/dev.c
index 35dfb83..aadf09b 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2208,7 +2208,7 @@ static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q,
 	return rc;
 }
 
-static DEFINE_PER_CPU(int, xmit_recursion);
+DEFINE_PER_CPU(int, xmit_recursion);
 #define RECURSION_LIMIT 10
 
 /**


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html