netdev - Re: XPS configuration question (on tg3)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <1473293634.15733.14.camel@edumazet-glaptop3.roam.corp.google.com>
Date:   Wed, 07 Sep 2016 17:13:54 -0700
From:   Eric Dumazet <eric.dumazet@...il.com>
To:     Michal Soltys <soltys@....info>
Cc:     Alexander Duyck <alexander.duyck@...il.com>,
        Linux Netdev List <netdev@...r.kernel.org>
Subject: Re: XPS configuration question (on tg3)

On Thu, 2016-09-08 at 01:45 +0200, Michal Soltys wrote:
> > Keep in mind that TCP stack can send packets, responding to incoming
> > ACK.
> > 
> > So you might check that incoming ACK are handled by the 'right' cpu.
> > 
> > Without RFS, there is no such guarantee.

> Did some more testing today, indeed RFS helped with TCP flows, but ....
> it got me wondering:
> 
> In scenario such as: XPS off, RFS/RPS off, irqs pinned, process
> transfering data pinned - one tx queue was chosen (through hash) and
> consistently persisted throughout the whole transfer. No exceptions, at
> least none in the tests I did.

It depends if at least one packet for each flow sits in a qdisc/NIC
queue. TCP has this Out Of Order transmit logic preventing a TX queue
change, even if the process doing the sendmsg() is migrated.

git grep -n ooo_okay

> 
> When XPS is getting enabled, the only thing that changes is that instead
> of using hash to select one of available queues, the cpu running process
> is specifically told which queue it can use (and eventually selects one
> through hash if more than one is available). Shouldn't the choice
> persist throughout the transfer as well then ?

Sure, if the process doing the sendmsg() sticks to one cpu, and this cpu
is the one handling incoming ACK packets as well.