Message-Id: <20070822085013.d20e1fd4.randy.dunlap@oracle.com>
Date:	Wed, 22 Aug 2007 08:50:13 -0700
From:	Randy Dunlap <randy.dunlap@...cle.com>
To:	Krishna Kumar <krkumar2@...ibm.com>
Cc:	johnpol@....mipt.ru, herbert@...dor.apana.org.au, hadi@...erus.ca,
	kaber@...sh.net, shemminger@...ux-foundation.org,
	davem@...emloft.net, jagana@...ibm.com, Robert.Olsson@...a.slu.se,
	peter.p.waskiewicz.jr@...el.com, kumarkr@...ux.ibm.com,
	xma@...ibm.com, gaagaan@...il.com, netdev@...r.kernel.org,
	rdreier@...co.com, rick.jones2@...com, mcarlson@...adcom.com,
	jeff@...zik.org, general@...ts.openfabrics.org, mchan@...adcom.com,
	tgraf@...g.ch, sri@...ibm.com
Subject: Re: [PATCH 1/10 Rev4] [Doc] HOWTO Documentation for batching

On Wed, 22 Aug 2007 13:58:58 +0530 Krishna Kumar wrote:

> Add Documentation describing batching skb xmit capability.
> 
> Signed-off-by: Krishna Kumar <krkumar2@...ibm.com>
> ---
>  batching_skb_xmit.txt |   78 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 78 insertions(+)
> 
> diff -ruNp org/Documentation/networking/batching_skb_xmit.txt new/Documentation/networking/batching_skb_xmit.txt
> --- org/Documentation/networking/batching_skb_xmit.txt	1970-01-01 05:30:00.000000000 +0530
> +++ new/Documentation/networking/batching_skb_xmit.txt	2007-08-22 10:21:19.000000000 +0530
> @@ -0,0 +1,78 @@
> +		 HOWTO for batching skb xmit support
> +		 -----------------------------------
> +
> +Section 1: What is batching skb xmit
> +Section 2: How batching xmit works vs the regular xmit
> +Section 3: How drivers can support batching
> +Section 4: How users can work with batching
> +
> +
> +Introduction: Kernel support for batching skb
> +----------------------------------------------
> +
> +A new capability to support xmit of multiple skbs is provided in the netdevice
> +layer. Drivers which enable this capability should be able to process multiple
> +skbs in a single call to their xmit handler.
> +
> +
> +Section 1: What is batching skb xmit
> +-------------------------------------
> +
> +	This capability is optionally enabled by a driver by setting the
> +	NETIF_F_BATCH_SKBS bit in dev->features. The pre-requisite for a

                                                     prerequisite

> +	driver to use this capability is that it should have a reasonably

	I would say "reasonably-sized".

> +	sized hardware queue that can process multiple skbs.
> +
> +
> +Section 2: How batching xmit works vs the regular xmit
> +-------------------------------------------------------
> +
> +	The network stack gets called from upper layer protocols with a single
> +	skb to transmit. This skb is first enqueue'd and an attempt is made to

                                           enqueued

> +	transmit it immediately (via qdisc_run). However, events like tx lock
> +	contention, tx queue stopped, etc, can result in the skb not getting

                                      etc.,

> +	sent out and it remains in the queue. When the next xmit is called or
> +	when the queue is re-enabled, qdisc_run could potentially find
> +	multiple packets in the queue, and iteratively send them all out
> +	one-by-one.
> +
> +	Batching skb xmit is a mechanism to exploit this situation where all
> +	skbs can be passed in one shot to the device. This reduces driver
> +	processing, locking at the driver (or in stack for ~LLTX drivers)
> +	gets amortized over multiple skbs, and in case of specific drivers
> +	where every xmit results in a completion processing (like IPoIB) -
> +	optimizations can be made in the driver to request a completion for
> +	only the last skb that was sent which results in saving interrupts
> +	for every (but the last) skb that was sent in the same batch.
> +
> +	Batching can result in significant performance gains for systems that
> +	have multiple data stream paths over the same network interface card.
> +
> +
> +Section 3: How drivers can support batching
> +---------------------------------------------
> +
> +	Batching requires the driver to set the NETIF_F_BATCH_SKBS bit in
> +	dev->features.
> +
> +	The driver's xmit handler should be modified to process multiple skbs
> +	instead of one skb. The driver's xmit handler is called either with a

                                                                           an

> +	skb to transmit or NULL skb, where the latter case should be handled
> +	as a call to xmit multiple skbs. This is done by sending out all skbs
> +	in the dev->skb_blist list (where it was added by the core stack).
> +
> +
> +Section 4: How users can work with batching
> +---------------------------------------------
> +
> +	Batching can be disabled for a particular device, e.g. on desktop
> +	systems if only one stream of network activity for that device is
> +	taking place, since performance could be slightly affected due to
> +	extra processing that batching adds (unless packets are getting
> +	sent fast resulting in stopped queue's). Batching can be enabled if

                                       queues).

> +	more than one stream of network activity per device is being done,
> +	e.g. on servers; or even desktop usage with multiple browser, chat,
> +	file transfer sessions, etc.
> +
> +	Per device batching can be enabled/disabled by passing 'on' or 'off'
> +	respectively to ethtool.

	with what other parameter(s), e.g.,

	ethtool <dev> batching on/off ?

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists