netdev - Re: CPU scheduler to TXQ binding? (ixgbe vs. igb)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1411047209.7106.255.camel@edumazet-glaptop2.roam.corp.google.com>
Date:	Thu, 18 Sep 2014 06:33:29 -0700
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Jesper Dangaard Brouer <jbrouer@...hat.com>
Cc:	Alexander Duyck <alexander.h.duyck@...el.com>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	Tom Herbert <therbert@...gle.com>
Subject: Re: CPU scheduler to TXQ binding? (ixgbe vs. igb)

On Thu, 2014-09-18 at 08:56 +0200, Jesper Dangaard Brouer wrote:

> After setting up XPS to CPU 1:1 binding, it works most of the time.
> Meaning, most of the traffic will go through the TXQ I've bound the
> process to, BUT some packets can still choose another TXQ (observed
> monitoring tc output and blqmon).

Note that for TCP, there are packets sent by the process doing the
send(), or packets sent by cpu doing TX completion (because of TSQ),
but also packets sent by ACK processing done in the reverse way.

As Alexander explained, if the ACK packets are delivered into another
CPU, then you might select another TX queue.

This is mostly prevented because of ooo_okay logic, meaning that a busy
bulk flow should stick into a single TX queue, no matter of XPS says.

A TCP_RR workload is free to chose whatever queue, because every packet
starting a RR block has the ooo_okay set (As prior data was delivered
and acknowledged by the opposite peer)

> 
> Could this be related to the missing RPS setup?

No, for this to really work, you need hardware support, so that ACK
packets take the same RX queue than the sent packets.

> 
> Can I get some hints setting up RPS?
> 

Documentation/networking/scaling.txt  is full of hints...

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html