Message-Id: <1181904378.4121.51.camel@moonstone.uk.level5networks.com>
Date:	Fri, 15 Jun 2007 11:46:18 +0100
From:	Kieran Mansley <kmansley@...arflare.com>
To:	xen-devel@...ts.xensource.com
Cc:	netdev@...r.kernel.org, herbert@...dor.apana.org.au
Subject: [PATCH 0/4] [Net] Support accelerated network plugin modules

This is a repost of some earlier patches to the xen-devel mailing list,
with a number of changes thanks to some useful suggestions from others.

I've also CC'd netdev@...r.kernel.org at Herbert Xu's request as some of
the files being patched may be merged into upstream linux soon, and so
folks there may have
opinions too.

Changes from last time:
 - Improved the link between frontend and accelerated plugin so that
netif_wake_queue is only called when both have available TX slots.
Thanks to Herbert Xu for pointing this out.
 - Removed macro that safely called the accelerated hooks; this code is
now inline in the relevant places. 
 - Some small cleanup modifications, and resync patches to latest xen-
unstable.hg, including source tree layout changes for linux-2.6.18-
xen.hg

The discussion following the previous posting of these patches was
mainly concerned with the approach to locking in the frontend.  Some
felt it was too complex and suggested using RCU instead.  However, I
have been unable to convince myself that RCU would offer suitable
protection against the accelerated plugin module unloading in the middle
of a hook call, particularly where these hook calls might block.  The
existing locking has been well tested and is therefore known to work.
In addition, the complexity is largely to ensure there is low locking
overhead on the data path, and so a change to simplify it would not
likely increase performance.  As a result the locking has not changed
substantially since the last patch.  However if others feel strongly
about this and can convince me that RCU would be adequate, I wouldn't
object to making this change.

What follows is the text from the original post providing some
background on the patches:

This set of patches provides the hooks and support necessary for
accelerated network plugin modules to attach to Xen's netback and
netfront.  These modules provide a fast path for network traffic where
there is hardware support available for the netfront driver to send and
receive packets directly to a NIC (such as those available from
Solarflare).

As there are currently no available plugins, I've attached a couple of
dummy ones to illustrate how the hooks could be used.  These are
incomplete (and clearly wouldn't even compile) in that they only include
code to show the interface between the accelerated module and
netfront/netback.  A lot of the comments hint at what code should go
where.  They don't show any interface between the accelerated frontend
and accelerated backend, or hardware access, for example, as those would
both be specific to the implementation.  I hope they help illustrate
this, but if you have any questions I'm happy to provide more
information.

A brief overview of the operation of the plugins:  When the accelerated
modules are loaded, a VI is created by the accelerated backend to allow
the accelerated frontend to safely access portions of the NIC.  For RX,
when packets are received by the accelerated backend, it will examine
them and if appropriate insert filters into the NIC to deliver future
packets on that address directly to the accelerated frontend's VI.  For
TX, netfront gives each accelerated frontend the option of sending each
packet, which it can accept (if it wants to send it directly to the
hardware) or decline (if it thinks this is more appropriate to send via
the normal network path).

We have found that using this approach to accelerating network traffic,
domU to domU connections (across the network) can achieve close to the
performance of dom0 to dom0 connections on a 10Gbps ethernet.  This is
roughly double the bandwidth seen with unmodified Xen.


View attachment "dummy_accel_backend.c" of type "text/x-csrc" (3325 bytes)

View attachment "dummy_accel_frontend.c" of type "text/x-csrc" (9658 bytes)

Powered by blists - more mailing lists