[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <1237018830.4966.414.camel@flash>
Date: 14 Mar 2009 00:20:42 -0800
From: Ramkrishna Vepa <ram.vepa@...erion.com>
To: Netdev <netdev@...r.kernel.org>,
David Miller <davem@...emloft.net>,
Jeff Garzik <jgarzik@...ox.com>
Cc: Ramkrishna Vepa <ram.vepa@...erion.com>
Subject: [net-2.6 PATCH 1/10] Neterion: New driver: Driver help file
- vxge driver help text file.
Signed-off-by: Sivakumar Subramani <sivakumar.subramani@...erion.com>
Signed-off-by: Rastapur Santosh <santosh.rastapur@...erion.com>
Signed-off-by: Ramkrishna Vepa <ram.vepa@...erion.com>
---
diff -urpN org/Documentation/networking/vxge.txt patch_1/Documentation/networking/vxge.txt
--- org/Documentation/networking/vxge.txt 1969-12-31 16:00:00.000000000 -0800
+++ patch_1/Documentation/networking/vxge.txt 2009-03-12 11:23:12.000000000 -0700
@@ -0,0 +1,476 @@
+Release notes for Neterion's (Formerly S2io) X3100 Linux Driver.
+
+contents
+--------
+
+1) Introduction
+2) Load/Unload Neterion driver
+3) Identifying the adapter/interface
+4) Features supported
+5) Configurable driver parameters
+6) Performance Suggestions
+7) Transmit performance
+8) Receive performance
+9) Troubleshooting
+10) Ethtool Support
+11) Known Issues
+12) Common Problems
+13) Available Downloads
+
+1) Introduction:
+----------------
+Linux Driver for Neterion 10 Gigabit Ethernet PCI Express I/O Virtualized Server Adapter
+
+ * This driver supports all Neterion's X3100 10GbE adapters and 2.4.x and 2.6.x kernels.
+ * Supports several features such as jumbo frames, MSI-X, checksum offloads, TSO, LRO and so on.
+(See below for complete list of features supported for both IPv4 and IPv6)
+
+2) Load/Unload Neterion driver:
+-------------------------------
+
+i). Load driver
+ For 2.4 kernel
+ # insmod /lib/modules/<kernel_version>/kernel/drivers/net/vxge.o
+ For 2.6 kernel
+ # insmod /lib/modules/<kernel_version>/kernel/drivers/net/vxge.ko
+ Using modprobe command:
+ # modprobe vxge
+
+ Verify driver loaded using lsmod
+ # lsmod |grep vxge
+
+ii). Configure IP address
+ ifconfig eth# <IP_address> up
+ # ifconfig eth1 17.1.1.100 up
+
+iii). Modify MTU
+ ifconfig eth# mtu <new_mtu>
+ # ifconfig eth1 mtu 9000
+
+iv). Unload driver
+ # ifconfig eth# down
+ # rmmod vxge
+
+3) Identifying the adapter/interface:
+-------------------------------------
+
+i). Insert the adapter(s) in your system.
+
+ii). Load driver
+ # insmod /lib/modules/<kernel_version>/kernel/drivers/net/vxge.ko
+
+ Using modprobe command:
+ # modprobe vxge
+
+iii). View log messages
+ # dmesg | tail -40
+
+ Messages similar to the following will output which identify the adapter
+ type, adapter revision, driver version, interface number, and interrupt type:
+
+ eth2: Neterion 10 Gigabit Ethernet X3110 Rev (2)
+ eth2: MAC ADDR: 00:0C:FC:00:B2:53 SERIAL NUMBER: SXJ0839036
+ eth2: Firmware version: 0.0.34 Date : 12/02/2008
+ eth2: Driver version: v.1.0.0 kernel version 2.6.16
+ eth2: Build time: Nov 25 2008, 23:19:49
+ eth2: 8 Vpath(s) enabled
+ eth2: Interrupt type MSI-X
+ eth2: RTH steering enabled for TCP_IPV4
+ eth2: Tx port steering enabled
+ eth2: Large receive offload enabled
+ eth2: Multiqueue support disabled
+ eth2: VLAN tag stripping disabled
+ eth2: NAPI disabled
+ eth2: Ring blocks : 2
+ eth2: Fifo blocks : 12
+
+iv). To associate an interface with a physical adapter use "ethtool -p eth#".
+ The corresponding adapter's LED will blink until canceled (ctrl+c).
+
+
+4) Features supported:
+----------------------
+
+i) I/O Virtualization Offloads:
+ - Single function mode
+ - Multi function mode
+
+ii) PCI-SIG's I/O Virtualization:
+ - Single Root mode: v1.0
+ - Multi-Root mode: v1.0
+
+iii) Jumbo frames:
+ X3100 Series supports MTU up to 9600 bytes, modifiable using
+ ifconfig command.
+
+iv) Offloads supported:
+ Checksum offload (TCP/UDP/IP) on transmit and receive paths
+ TCP Segmentation Offload (TSO) on transmit path
+ Large Receive Offload (LRO, TCP aggregation) on receive path
+
+v) MSI-X:
+ Can be enabled on platforms which support it, resulting in noticeable
+ performance improvement (up to 7% on certain platforms).
+
+vi) NAPI:
+ For better Rx interrupt moderation.
+
+vii) RTH (Receive Traffic Hash):
+ Receive side steering for better scaling.
+
+viii) Statistics:
+ Comprehensive MAC-level and software statistics displayed using
+ "ethtool -S" option.
+
+ix) Multi-Vpaths:
+ Up to 17 hardware based transmit and receive data channels, with
+ multiple steering options.
+
+
+5) Configurable driver parameters:
+----------------------------------
+
+i) intr_type
+ Specifies interrupt type.
+ Valid range: 0, 2 (INTA, MSI-X respectively)
+ Default: 2
+
+ii) lro
+ Enables/disables Large Receive Offload (LRO).
+ Valid range: 0,1 (disabled, enabled respectively)
+ Default: 1
+
+iii) lro_max_bytes
+ Specifies maximum number of bytes to be aggregated as
+ a single large packet.
+ Valid range: 0-FFFF
+ Default: 0x4000
+
+iv) max_config_dev
+ Specifies maximum device functions to be enabled.
+ Valid range: 1-8
+ Default:
+
+v) max_config_port:
+ Specifies number of ports to be enabled.
+ Valid range: 0,1
+ Default: 1
+
+vi) max_config_vpath:
+ Specifies maximum VPATH(s) configured for each device function.
+ Valid range: 1-17
+ Default:
+
+vii) napi
+ Enables/disables NAPI support.
+ Automatically disabled when intr_type is set to MSI-X on kernels below
+ 2.6.24.
+ Valid range: 0,1 (disabled, enabled respectively)
+ Default: 1
+
+viii) rpa_strip_vlan_tag
+ Enables/disables vlan tag stripping from all received tagged frames that
+ are not replicated at the internal L2 switch.
+ Valid range: 0,1 (disabled, enabled respectively)
+ Default: 1
+
+ix) tx_steering_type
+ Specifies criteria for steering transmitted packets.
+ Valid range: 0-4 (disabled, priority, VLAN ID, multiq and L4 port respectively)
+ Default: 4
+
+The following parameters are used collectively to enable the receive
+traffic steering capability of the driver.
+
+x) addr_learn_en
+ Enable Receive Traffic Steering using MAC destination address.
+ Valid range: 0,1 (disabled, enabled respectively)
+ Default: 0
+
+xi) rx_steering_type
+ Specifies criteria for steering received packets.
+ Valid range: 0-4 (disabled, L4 port, RTH, priority and VLAN ID respectively)
+ Default: 2 (when system supports MSI-X and has more than 4 CPUs)
+
+xii) ring_indicate_max_pkts
+ Sets maximum number of received frames to be processed within single interrupt.
+ Valid range: 1 - 65536
+ Default : 128
+
+xiii) fifo_indicate_max_pkts:
+ Sets maximum number of transmit frames to be processed within single interrupt.
+ Valid range: 1 - 65536
+ Default : 32
+
+xiv) ring_blocks:
+ Numbers of RxD blocks in the ring
+ Valid range: 1 - 128
+ Default : 2
+
+xv) Boot time configuration
+ In order to load option(s) at boot time, add the option(s) to
+ /etc/modprobe.conf.
+ Example:
+ alias eth# vxge
+ options vxge rx_blocks=4 rx_steering_type=2
+
+
+6) Performance Suggestions:
+---------------------------
+
+- Set MTU to maximum - 9000 for switch setup, 9600 for back-to-back.
+For MTU 1500:
+-------------
+### IPV4 specific settings
+# turns TCP timestamp support off, default 1, reduces CPU use
+$sysctl -w net.ipv4.tcp_timestamps = "0"
+# turn SACK support off, default on
+$sysctl -w net.ipv4.tcp_sack="0"
+# on systems with a VERY fast bus -> memory interface this is the big gainer
+# sets min/default/max TCP read buffer, default 4096 87380 174760
+$sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
+# sets min/pressure/max TCP write buffer, default 4096 16384 131072
+$sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
+# sets min/pressure/max TCP buffer space, default 31744 32256 32768
+$sysctl -w net.ipv4.tcp_mem="210000 210000 210000"
+
+### CORE settings (mostly for socket and UDP effect)
+# maximum receive socket buffer size, default 131071
+$sysctl -w net.core.rmem_max="524287"
+# maximum send socket buffer size, default 131071
+$sysctl -w net.core.wmem_max="524287"
+# default receive socket buffer size, default 65535
+$sysctl -w net.core.rmem_default="524287"
+# default send socket buffer size, default 65535
+$sysctl -w net.core.wmem_default="524287"
+# maximum amount of option memory buffers, default 10240
+$sysctl -w net.core.optmem_max="524287"
+# number of unprocessed input packets before kernel starts dropping them, default 300
+$sysctl -w net.core.netdev_max_backlog="300000"
+
+For MTU 9000:
+-------------
+### IPV4 specific settings
+# turns TCP timestamp support off, default 1, reduces CPU use
+$sysctl -w net.ipv4.tcp_timestamps = "0"
+# turn SACK support off, default on
+$sysctl -w net.ipv4.tcp_sack="0"
+# on systems with a VERY fast bus -> memory interface this is the big gainer
+# sets min/default/max TCP read buffer, default 4096 87380 174760
+$sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
+# sets min/pressure/max TCP write buffer, default 4096 16384 131072
+$sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"
+# sets min/pressure/max TCP buffer space, default 31744 32256 32768
+$sysctl -w net.ipv4.tcp_mem="10000000 10000000 10000000"
+
+### CORE settings (mostly for socket and UDP effect)
+# maximum receive socket buffer size, default 131071
+$sysctl -w net.core.rmem_max="524287"
+# maximum send socket buffer size, default 131071
+$sysctl -w net.core.wmem_max="524287"
+# default receive socket buffer size, default 65535
+$sysctl -w net.core.rmem_default="524287"
+# default send socket buffer size, default 65535
+$sysctl -w net.core.wmem_default="524287"
+# maximum amount of option memory buffers, default 10240
+$sysctl -w net.core.optmem_max="524287"
+# number of unprocessed input packets before kernel starts dropping them, default 300
+$sysctl -w net.core.netdev_max_backlog="300000"
+
+7) Transmit performance:
+------------------------
+
+- Ensure Transmit Checksum offload and TSO are enabled.
+ Use ethtool to verify/set these parameters.
+
+ # ethtool -k eth#
+ Offload parameters for eth0:
+ rx-checksumming: on
+ tx-checksumming: on
+ scatter-gather: on
+ tcp segmentation offload: off
+
+- Turn on Transmit Checksum offload and TSO:
+ # ethtool -K eth# tx on
+ # ethtool -K eth# tso on (for 2.6 kernels)
+
+
+8) Receive performance:
+-----------------------
+
+- Ensure Receive Checksum offload is enabled. Use ethtool to verify/set.
+ # ethtool -k eth#
+ # ethtool -K eth# rx on
+
+- If MTU is set to 1500, receive performance can be improved by increasing
+ the default TCP window size and enabling LRO (see "Configurable driver
+ parameters" section).
+ # sysctl -p sysctl_neterion_1500.conf
+
+- Enable NAPI to bring down CPU utilization.
+
+- If CPU bottleneck is being hit (close to 0% idle), you can play around with
+ utilization parameters by using tune_driver script. This may help bring
+ down CPU utilization and improve throughput. See "Driver tuning" section.
+
+
+9) Troubleshooting:
+-------------------
+
+For X3100 adapter assigned eth2
+
+Statistics Dump:
+ # ethtool -S eth2
+
+Register Dump:
+ # ethtool -d eth2
+
+
+Before Contacting Neterion for Support:
+
+i) Please contact your network adapter provider (i.e. OEM or reseller) for
+ initial support.
+
+ii) Run the titanl_dump utility which is included in the linux driver package on
+ Nterion's driver downloads page.
+ # titanl_dump eth#
+
+iii) When contacting Neterion for support, please include the following in
+ your email to support@...erion.com:
+ - The titanl_dump.log.gz file resulting from step 2 above
+ - Brief description of your setup and how you are using
+ your X3100 adapter
+
+
+
+10) Ethtool Support:
+--------------------
+Ethtool is useful for driver configuration, diagnostics and displaying
+statistical information. The latest version of Ethtool is available at
+http://sourceforge.net/projects/gkernel/
+
+ ethtool eth#
+ Displays current device settings
+
+ ethtool -a eth#
+ Displays pause parameter information.
+
+ ethtool -A eth#
+ Changes pause parameters.
+
+ ethtool -d eth#
+ Retrieves and prints a register dump for the specified
+ ethernet device.
+
+ ethtool -e eth# [raw on|off] [offset N] [length N]
+ Retrieves and prints an EEPROM dump for the specified
+ ethernet device. When raw is enabled, then it dumps the raw
+ EEPROM data to stdout. The length and offset parameters allow
+ dumping certain portions of the EEPROM. Default is to dump
+ the entire EEPROM.
+
+ ethtool -E eth# [magic N] [offset N] [value N]
+ Changes EEPROM byte for the specified ethernet device.
+ Offset and value specify which byte and its new value.
+ Because of the persistent nature of writing to the EEPROM, a
+ device-specific magic key must be specified to prevent the
+ accidental writing to the EEPROM.
+
+ ethtool -g ethx
+ Queries the specified ethernet device for rx/tx
+ ring parameter information.
+
+ ethtool -i eth#
+ Displays device driver information
+
+ ethtool -k eth#
+ Displays offload parameters information.
+
+ ethtool -K eth# [rx on|off] [tx on|off] [sg on|off] [tso on|off]
+ Changes offload parameters.
+ rx - RX checksum offload
+ tx - TX checksum offload
+ sg - scatter-gather
+ tso - TCP Segmentation Offload
+
+ ethtool -p eth# [N]
+ Initiates adapter-specific action intended to enable an
+ operator to easily identify the physical adapter by sight.
+ Typically this involves blinking one or more LEDs on the
+ specified ethernet port.
+
+ N - Length of time to perform phys-id, in seconds.
+
+ ethtool -S eth#
+ Displays adapter and driver statistics.
+
+ ethtool -t eth# [offline|online]
+ Executes adapter selftest on the specified ethernet device.
+ Possible test modes are:
+
+ offline (default) - performs full set of tests, possibly
+ interrupting normal adapter operation during the tests.
+ online - performs limited set of tests, not interrupting normal
+ adapter operation.
+
+11) Known Issues:
+-----------------
+i) TSO disabled in SUSE 9:
+ TSO is disabled by default in SuSE 9. It may be manually enabled
+ using the ethtool command.
+ # ethtool -K eth# tso on
+
+12) Common Problems:
+--------------------
+
+i) Symbolic link:
+ Before debugging any compilation related problems, ensure that the
+ following symbolic link/directory points to the correct kernel source
+ tree.
+ For 2.4, /usr/src/linux-2.4
+ For 2.6, /usr/src/linux-2.6
+
+ii) Version Mismatch:
+ One of the common problems encountered on trying to load the driver is
+ related to a version mismatch between current kernel and the kernel for
+ which the driver was built. Typical error message is as follows:
+
+ Using /lib/modules/2.x.y-SMP/kernel/drivers/net/vxge.o
+ /lib/modules/2.x.y-SMP/kernel/drivers/net/vxge.o: kernel-module
+ version mismatch /lib/modules/2.x.y-SMP/kernel/drivers/net/vxge.o
+ was compiled for kernel version 2.x.y while this kernel is version
+ 2.x.y-SMP.
+
+ To fix this problem, ensure the kernel is enabled for SMP by following
+ these steps:
+
+ - Go to the directory where kernel sources are located.
+ (e.g. /usr/src/linux-2.4.19.SuSE)
+
+ - Execute "make menuconfig". This will bring up the "Linux
+ Kernel Configuration" menu. Select "Processor type
+ and features". Ensure that "Symmetric multi-processing
+ support" is enabled.
+ Save this configuration and exit.
+
+ This will compile the kernel with SMP support.
+
+13) Available Downloads:
+------------------------
+
+ For latest available Neterion drivers or further support please
+ contact your network adapter provider or check the Neterion support
+ download site at
+ http://www.neterion.com/support/xframe_customers.html
+
+ This readme applies to the Neterion Linux driver available from our
+ website. Although the Neterion Linux drivers are included in the
+ mainline kernel and major Linux distributions (i.e. Red Hat, Suse, etc),
+ and are similar, each distribution accepts or rejects our driver changes
+ at their own discretion. Therefore, some features may be enabled or
+ disabled in different releases.
+
+===============================================================================
+ Neterion Inc. Proprietary
+ COPYRIGHT (c) 2002-2009 Neterion Inc, ALL RIGHTS RESERVED
+
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists