netdev - Re: setsockopt()

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 09 Jul 2008 11:10:31 -0700
From:	Roland Dreier <rdreier@...co.com>
To:	Bill Fink <billfink@...dspring.com>
Cc:	Evgeniy Polyakov <johnpol@....mipt.ru>,
	David Miller <davem@...emloft.net>, aglo@...i.umich.edu,
	shemminger@...tta.com, netdev@...r.kernel.org, rees@...ch.edu,
	bfields@...ldses.org
Subject: Re: setsockopt()

 > The other weird thing about your test is the huge difference in
 > the receiver (and server in this case) CPU utilization between the
 > autotuning and explicit setting cases (2 %RX versus 96 %RX).

I think I found another clue -- it seems that CPU affinity has something
to do with the results.  Usually I pin the adapter interrupt to CPU 0
and use "taskset 4" to pin the benchmarking process to CPU 2 (this leads
to the best performance for these particular systems in almost all
benchmarks).  But with nuttcp I see the following:

  with taskset 4:

    $ taskset 4 nuttcp -T30 192.168.145.73 
     9911.3125 MB /  30.01 sec = 2770.3202 Mbps 42 %TX 10 %RX
    $ taskset 4 nuttcp -w128k -T30 192.168.145.73 
    36241.9375 MB /  30.00 sec = 10133.8931 Mbps 89 %TX 96 %RX

  with no taskset (ie let kernel schedule as it wants to):

    $ nuttcp -T30 192.168.145.73
    36689.6875 MB /  30.00 sec = 10259.1525 Mbps 99 %TX 96 %RX
    $ nuttcp -w128k -T30 192.168.145.73
    36486.0000 MB /  30.00 sec = 10202.1870 Mbps 74 %TX 95 %RX

so somehow setting the window helps with the scheduling of
processes... I guess autotuning lets some queue get too long or
something like that.  The actual window doesn't matter too much, since
the delay of the network is low enough that even though the bandwidth is
very high, the BDP is quite small.  (With a 25 usec RTT, a 128 KB window
should be enough for 40 Gbps, well over the raw link speed of 16 Gbps
that I have)

 - R.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html