[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 02 Feb 2009 16:09:33 -0500
From: Wes Chow <wchow@...enacr.com>
To: Eric Dumazet <dada1@...mosbay.com>
CC: netdev@...r.kernel.org
Subject: Re: Multicast packet loss
Eric Dumazet wrote:
> Wes Chow a écrit :
>> (I'm Kenny's colleague, and I've been doing the kernel builds)
>>
>> First I'd like to note that there were a lot of bnx2 NAPI changes between
>> 2.6.21 and 2.6.22. As a reminder, 2.6.21 shows tiny amounts of packet loss,
>> whereas loss in 2.6.22 is significant.
>>
>> Second, some CPU affinity info: if I do like Eric and pin all of the
>> apps onto a single CPU, I see no packet loss. Also, I do *not* see
>> ksoftirqd show up on top at all!
>>
>> If I pin half the processes on one CPU and the other half on another CPU, one
>> ksoftirqd processes shows up in top and completely pegs one CPU. My packet loss
>> in that case is significant (25%).
>>
>> Now, the strange case: if I pin 3 processes to one CPU and 1 process to
>> another, I get about 25% packet loss and ksoftirqd pins one CPU. However, one
>> of the apps takes significantly less CPU than the others, and all apps lose the
>> *exact same number of packets*. In all other situations where we see packet
>> loss, the actual number lost per application instance appears random.
>
> You see same number of packet lost because they are lost at NIC level
Understood.
I have a new observation: if I pin processes to just CPUs 0 and 1, I see
no packet loss. Pinning to 0 and 2, I do see packet loss. Pinning 2 and
3, no packet loss. 4 & 5 - no packet loss, 6 & 7 - no packet loss. Any
other combination appears to produce loss (though I have not tried all
28 combinations, this seems to be the case).
At first I thought maybe it had to do with processes pinned to the same
CPU, but different cores. The machine is a dual quad core, which means
that CPUs 0-3 should be a physical CPU, correct? Pinning to 0/2 and 0/3
produce packet loss.
I've also noticed that it does not matter which of the working pairs I
pin to. For example, pinning 5 processes in any combination on either
0/1 produce no packet loss, pinning all 5 to just CPU 0 also produces no
packet loss.
The failures are also sudden. In all of the working cases mentioned
above, I don't see ksoftirqd on top at all. But when I run 6 processes
on a single CPU, ksoftirqd shoots up to 100% and I lose a huge number of
packets.
>
> Normaly, softirq runs on same cpu (the one handling hard irq)
What determines which CPU the hard irq occurs on?
Wes
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists