netdev - Re: [PATCH v3 05/10] bql: Byte queue limits

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1322064019.17693.53.camel@edumazet-HP-Compaq-6005-Pro-SFF-PC>
Date:	Wed, 23 Nov 2011 17:00:19 +0100
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Tom Herbert <therbert@...gle.com>
Cc:	davem@...emloft.net, netdev@...r.kernel.org
Subject: Re: [PATCH v3 05/10] bql: Byte queue limits

Le mardi 22 novembre 2011 à 21:52 -0800, Tom Herbert a écrit :
> Networking stack support for byte queue limits, uses dynamic queue
> limits library.  Byte queue limits are maintained per transmit queue,
> and a dql structure has been added to netdev_queue structure for this
> purpose.
> 
> Configuration of bql is in the tx-<n> sysfs directory for the queue
> under the byte_queue_limits directory.  Configuration includes:
> limit_min, bql minimum limit
> limit_max, bql maximum limit
> hold_time, bql slack hold time
> 
> Also under the directory are:
> limit, current byte limit
> inflight, current number of bytes on the queue
> 
> Signed-off-by: Tom Herbert <therbert@...gle.com>
> ---
>  include/linux/netdevice.h |   28 ++++++++
>  net/Kconfig               |   13 ++++
>  net/core/dev.c            |    3 +
>  net/core/net-sysfs.c      |  150 ++++++++++++++++++++++++++++++++++++++++++---
>  4 files changed, 186 insertions(+), 8 deletions(-)
> 
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 8b3eb8a..e17ece6 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -43,6 +43,7 @@
>  #include <linux/rculist.h>
>  #include <linux/dmaengine.h>
>  #include <linux/workqueue.h>
> +#include <linux/dynamic_queue_limits.h>
>  
>  #include <linux/ethtool.h>
>  #include <net/net_namespace.h>
> @@ -557,6 +558,9 @@ struct netdev_queue {
>  	 * please use this field instead of dev->trans_start
>  	 */
>  	unsigned long		trans_start;
> +#ifdef CONFIG_BQL
> +	struct dql		dql;
> +#endif
>  } ____cacheline_aligned_in_smp;
>  
>  static inline int netdev_queue_numa_node_read(const struct netdev_queue *q)
> @@ -1927,6 +1931,15 @@ static inline int netif_xmit_frozen_or_stopped(const struct netdev_queue *dev_qu
>  static inline void netdev_tx_sent_queue(struct netdev_queue *dev_queue,
>  					unsigned int pkts, unsigned int bytes)
>  {
> +#ifdef CONFIG_BQL
> +	dql_queued(&dev_queue->dql, bytes);
> +	if (unlikely(dql_avail(&dev_queue->dql) < 0)) {
> +		set_bit(__QUEUE_STATE_STACK_XOFF, &dev_queue->state);
> +		if (unlikely(dql_avail(&dev_queue->dql) >= 0))
> +			clear_bit(__QUEUE_STATE_STACK_XOFF,
> +			    &dev_queue->state);
> +	}
> +#endif
>  }
>  
>  static inline void netdev_sent_queue(struct net_device *dev,
> @@ -1938,6 +1951,18 @@ static inline void netdev_sent_queue(struct net_device *dev,
>  static inline void netdev_tx_completed_queue(struct netdev_queue *dev_queue,
>  					     unsigned pkts, unsigned bytes)
>  {
> +#ifdef CONFIG_BQL
> +	if (likely(bytes)) {
> +		dql_completed(&dev_queue->dql, bytes);
> +		if (unlikely(test_bit(__QUEUE_STATE_STACK_XOFF,
> +		    &dev_queue->state) &&
> +		    dql_avail(&dev_queue->dql) >= 0)) {

Maybe we can use some trick to avoid many wakeups ?

I feel that an other cpu might discover queue state is now XON and can
start xmit, without the extra __netif_schedule_queue() cost for this cpu
(softirq...). In a stress situation, cpu handling NIC interrupts can be
hogged...

The idea would be to clear STACK_XOFF bit, and if bytes are still in
flight (anotehr completion should come later), not call
netif_schedule_queue()

> +			if (test_and_clear_bit(__QUEUE_STATE_STACK_XOFF,
> +			     &dev_queue->state))
> +				netif_schedule_queue(dev_queue);
> +		}
> +	}
> +#endif
>  }
>  
>  static 



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html