Message-ID: <1349792241.21847.199.camel@zakaz.uk.xensource.com>
Date:	Tue, 9 Oct 2012 15:17:21 +0100
From:	Ian Campbell <Ian.Campbell@...rix.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
CC:	Eric Dumazet <edumazet@...gle.com>,
	xen-devel <xen-devel@...ts.xen.org>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	Konrad Rzeszutek Wilk <konrad@...nel.org>,
	Sander Eikelenboom <linux@...elenboom.it>
Subject: Re: compound skb frag pages appearing in start_xmit

On Tue, 2012-10-09 at 14:54 +0100, Eric Dumazet wrote:
> On Tue, 2012-10-09 at 14:47 +0100, Ian Campbell wrote:
> > Hi Eric,
> > 
> 
> Hi Ian
> 
> > Sander has discovered an issue where xen-netback is given a compound
> > page as one of the skb frag pages to transmit. Currently netback can
> > only handle PAGE_SIZE'd frags and bugs out.
> > 
> > I suspect this is something to do with 69b08f62e174 "net: use bigger
> > pages in __netdev_alloc_frag", although perhaps not because it looks
> > like only tg3 uses it and Sander has an r8169. Also tg3 seems to only
> > call netdev_alloc_frag for sizes < PAGE_SIZE. I'm probably missing
> > something.
> 
> 
> Its not the commit you want ;)
> 
> > 
> > Are all net drivers expected to be able to handle compound pages in the
> > frags? Obviously it is to their benefit to do so, so it is something
> > I'll want to look into for netback.
> > 
> 
> Not sure why a net driver would care of COMPOUND page at all ?
> 
> a Fragment has a struct page *, and a size.
> 
> a page can be order-0, order-1, order-2, order-3, ...

I keep falling into this trap that a struct page * can be order > 0.

The Xen PV interfaces deal in order-0 pages only. Also things which are
contiguous in physical space may not be contiguous in DMA space (which
we call "machine memory" in Xen terminology).

The first is probably a specific quirk of Xen, but I thought there were
other architectures where physical and DMA space we not necessarily
contiguous and which would therefore need special handling (I guess
those platforms all have IOMMUs)

> > I expect the main factor here is bridging/forwarding, since the
> > receiving NIC and its driver appear to support compound pages but the
> > outgoing NIC (netback in this case) does not.
> > 
> > I guess my question is should I be rushing to fix netback ASAP or should
> > I rather be looking for a bug somewhere which caused a frag of this type
> > to get as far as netback's start_xmit in the first place?
> > 
> > Or am I just barking up the wrong tree to start with?
> 
> 
> 
> The problem comes because of 
> 
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commit;h=5640f7685831e088fe6c2e1f863a6805962f8e81
> 
> And yes, we must find a way to cope with this problem in your driver,
> because you can also benefit from increase of performance once fixed ;)
> 
> And yes I can certainly help, as I am the author of this patch ;)

I think I can mostly deal with this in the same way netback deals with
large skb heads i.e. by busting the multipage page into individual 4096
page chunks.

Does the higher order pages effectively reduce the number of frags which
are in use? e.g if MAX_SKB_FRAGS is 16, then for order-0 pages you could
have 64K worth of frag data.

If we switch to order-3 pages everywhere then can the skb contain 512K
of data, or does the effective maximum number of frags in an skb reduce
to 2?

If it's the latter then I think fixing netback is simple, if it's the
former then I might need to think a bit harder.

Ian.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists