lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:	Thu, 3 Jul 2014 19:44:08 -0300
From:	Flavio Leitner <fbl@...hat.com>
To:	netdev@...r.kernel.org
Subject: RSS is not efficient when forwarding


Hi,

I have a simple router setup which forwards traffic from
one ixgbe 82599ES to another ixgbe of the same model.

kernel: 3.16.0-rc2-00262-ga921e2a

  p2p1: 192.168.155.1/24 is the gateway of the LAN
  p2p2: 192.168.156.1/24 is the gateway of the other LAN

While the ARP is resolving, I can see the packets being spread
among all the 8 queues (8 online CPUs) available and that is fine.

However, as soon as the TCP traffic starts, all streams are
merged to rx-queue-0 which overwhelms one single CPU, so the
total throughput is about 4Gbits/sec.

I can see the driver sending different skb->hash for each stream,
so it can't be the NIC.

Also, if I run a local http on the router, the skb->hash pattern
doesn't change, but the workload is spread among all CPUs.

debug output while reproducing all the streams on rx-queue-0:
[...]
[11685.885093] ixgbe_rx_skb:1713 skb(ffff880222a77200) hash: 0xC2AF4A27
[11685.891454] ixgbe_rx_skb:1713 skb(ffff880222a77200) hash: 0x8C5B749D
[11685.897820] ixgbe_rx_skb:1713 skb(ffff880222a77200) hash: 0xA33BA6D5
[11690.845032] net_ratelimit: 3276406 callbacks suppressed
[...]

Any idea why all TCP streams are going to a single rx queue?

Thanks,
fbl


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists