netdev - Re: [RFC/PATCH] sungem: Spring cleaning and GRO support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20110531.194115.486383514.davem@davemloft.net>
Date:	Tue, 31 May 2011 19:41:15 -0700 (PDT)
From:	David Miller <davem@...emloft.net>
To:	benh@...nel.crashing.org
Cc:	netdev@...r.kernel.org, ruediger.herbst@...glemail.com,
	bhamilton04@...il.com
Subject: Re: [RFC/PATCH] sungem: Spring cleaning and GRO support

From: Benjamin Herrenschmidt <benh@...nel.crashing.org>
Date: Tue, 31 May 2011 17:59:05 +1000

> Now the results .... on a dual G5 machine with a 1000Mb link, no
> measurable netperf difference on Rx and a 3% loss on Tx.
> 
> So taking the lock is the Tx path hurts...

It shouldn't.  You're replacing one lock with another, and in fact
because TX reclaim occurs in softirq context (and thus SKB freeing can
be done directly, instead of rescheduled to a softirq) it should be
faster.

And I think I see what the problem is:

> +	if (unlikely(netif_queue_stopped(dev) &&
> +		     TX_BUFFS_AVAIL(gp) > (MAX_SKB_FRAGS + 1))) {
> +		netif_tx_lock(dev);
> +		if (netif_queue_stopped(dev) &&
> +		    TX_BUFFS_AVAIL(gp) > (MAX_SKB_FRAGS + 1))
> +			netif_wake_queue(dev);
> +		netif_tx_unlock(dev);
> +	}
>  }

Don't use netif_tx_lock(), that has a loop and multiple atomics :-)

It's going to grab a special global TX lock, and then grab a lock for
TX queue zero, and finally set an atomic state bit in TX queue zero.

Take a look at the implementation in netdevice.h

It's a special "lock everything TX", a mechanism for multiqueue
drivers to shut quiesce all TX queue activity safely in one operation.

Instead, do something like:

	struct netdev_queue *txq = netdev_get_tx_queue(dev, 0);

	__netif_tx_lock(txq, smp_processor_id();
	...
	__netif_tx_unlock(txq);

and I bet your TX numbers improve a bit.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html