netdev - Re: bond + tc regression ?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1241635518.13702.37.camel@hazard2.francoudi.com>
Date:	Wed, 06 May 2009 21:45:18 +0300
From:	Vladimir Ivashchenko <hazard@...ncoudi.com>
To:	Eric Dumazet <dada1@...mosbay.com>
Cc:	netdev@...r.kernel.org
Subject: Re: bond + tc regression ?


On Wed, 2009-05-06 at 05:36 +0200, Eric Dumazet wrote:

> Ah, I forgot about one patch that could help your setup too (if using more than one
> cpu on NIC irqs of course), queued for 2.6.31

I have tried the patch. Didn't make a noticeable difference. Under 850
mbps HTB+sfq load, 2.6.29.1, four NICs / two bond ifaces, IRQ balancing,
the dual-core server has only 25% idle on each CPU.

What's interesting, the same 850mbps load, identical machine, but with
only two NICs and no bond, HTB+esfq, kernel 2.6.21.2 => 60% CPU idle.
2.5x overhead.

> (commit 6a321cb370ad3db4ba6e405e638b3a42c41089b0)
> 
> You could post oprofile results to help us finding other hot spots.
> 
> 
> [PATCH] net: netif_tx_queue_stopped too expensive
> 
> netif_tx_queue_stopped(txq) is most of the time false.
> 
> Yet its cost is very expensive on SMP.
> 
> static inline int netif_tx_queue_stopped(const struct netdev_queue *dev_queue)
> {
> 	return test_bit(__QUEUE_STATE_XOFF, &dev_queue->state);
> }
> 
> I saw this on oprofile hunting and bnx2 driver bnx2_tx_int().
> 
> We probably should split "struct netdev_queue" in two parts, one
> being read mostly.
> 
> __netif_tx_lock() touches _xmit_lock & xmit_lock_owner, these
> deserve a separate cache line.
> 
> Signed-off-by: Eric Dumazet <dada1@...mosbay.com>
> 
> 
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 2e7783f..1caaebb 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -447,12 +447,18 @@ enum netdev_queue_state_t
>  };
>  
>  struct netdev_queue {
> +/*
> + * read mostly part
> + */
>  	struct net_device	*dev;
>  	struct Qdisc		*qdisc;
>  	unsigned long		state;
> -	spinlock_t		_xmit_lock;
> -	int			xmit_lock_owner;
>  	struct Qdisc		*qdisc_sleeping;
> +/*
> + * write mostly part
> + */
> +	spinlock_t		_xmit_lock ____cacheline_aligned_in_smp;
> +	int			xmit_lock_owner;
>  } ____cacheline_aligned_in_smp;
>  
> 
-- 
Best Regards,
Vladimir Ivashchenko
Chief Technology Officer
PrimeTel PLC, Cyprus - www.prime-tel.com
Tel: +357 25 100100 Fax: +357 2210 2211


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html