[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <alpine.DEB.1.10.0906051444360.29918@gentwo.org>
Date: Fri, 5 Jun 2009 15:17:26 -0400 (EDT)
From: Christoph Lameter <cl@...ux-foundation.org>
To: Roland Dreier <rdreier@...co.com>
cc: David Miller <davem@...emloft.net>, netdev@...r.kernel.org,
yosefe@...taire.COM
Subject: Re: IPoIB: Fix multicast packet drops before join is complete
On Fri, 5 Jun 2009, Roland Dreier wrote:
> > ARP is tied to managing small chunks of information about the network
> > infrastructure. Buffering the first few and throwing the rest away is
> > appropriate there for what the ARP protocol intends to do.
>
> Yes, but what the IP stack is doing is queueing a few packets while ARP
> is pending and dropping all other packets until the destination ethernet
> address is resolved.
Usually the ARP resolution is already established so its not that big an
issue. In order for ARP resolution to matter we have to send to a
IP address that is not in the current ARP cache.
We ran a test here sending UDP unicast and indeed we get udp packet loss.
However, *no* initial packets made it at all to the destination. We
had an effective outage for 3 milliseconds of packets being
discarded before anything is received. However, once it starts packets
flow continually without a problem.
This is actually better than IPoIB there is nothing before
the initial traffic that arrives. In the IPoIB case 3 packets arrive
suggesting to the other side that the link is okay.
In the MC case a IGMP join must also be performed which likely takes much
longer and the switches will hold off traffic for awhile. We have a
mininum here of 8 milliseconds of those outages. Then the first 3 packet
arrive.
> > UDP multicasting can be used for streaming information. And right now the
> > IPoIB layer is dropping thousands of packets whenever there was a pause of
> > a few minutes or when a new multicast group is used and there is some
> > delay that the network need to reestablish the multicast route.
>
> Yes -- and the required IB multicast resolution seems like it is an L2
> thing precisely analogous to ARP. So I think the original IPoIB code
> actually was doing the right thing.
Then why dont the 1G NICs do the same?
Why are apps no longer working right when we move them from 1G to
IPoIB?
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists