netdev - Re: [PATCH v4 net-next 0/4] net: low latency Ethernet device polling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <519DF6D3.40705@mellanox.com>
Date:	Thu, 23 May 2013 14:00:35 +0300
From:	Alex Rosenbaum <alexr@...lanox.com>
To:	Eliezer Tamir <eliezer.tamir@...ux.intel.com>
CC:	Dave Miller <davem@...emloft.net>, <linux-kernel@...r.kernel.org>,
	<netdev@...r.kernel.org>,
	Jesse Brandeburg <jesse.brandeburg@...el.com>,
	Don Skidmore <donald.c.skidmore@...el.com>,
	<e1000-devel@...ts.sourceforge.net>,
	Willem de Bruijn <willemb@...gle.com>,
	Andi Kleen <andi@...stfloor.org>, HPA <hpa@...or.com>,
	Or Gerlitz <or.gerlitz@...il.com>,
	Eilon Greenstien <eilong@...adcom.com>,
	Eliezer Tamir <eliezer@...ir.org.il>
Subject: Re: [PATCH v4 net-next 0/4] net: low latency Ethernet device polling

Eliezer,

With AmirV's help we got this working on our NIC as well and it look 
nice. We too see the nice performance gain.

I tested with epoll and as expected there is no performance improvement. 
I don't think there is any point delaying this feature commit due to 
this fact. Future development should handle that.

I also tested LLS with different message rates.
Using sockperf you can set a ping send rate (--msp) and measure latency 
at different rates (I don't think netperf can do this).
In the financial trading sector, low latencies for 100mps are just as 
important as in 50Kmsp (or higher). The market orders go out at these 
low rates.
I noticed a penalty in the latency performance as I go lower in mps. I 
don't think it is related to the LLS code but it is more obvious then 
without it since you reach lower results.

These numbers are for sockperf TCP ping-pong at different msp for a 12 
byte payload.
I verified LLS hit counter was at 100% for all different message rates 
on both server and client side.
Machine is a x86_64 Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz with 64GB ram.

    rate   | LLS-on RTT | LLS-off RTT
   (msp) |    (usec)      |   (usec)
  10000 |     14.0        |     21.8
    1000 |     15.6        |     23.0
      100 |     16.6        |     24.4

You can see that as I go lower in send message rate the latency increases.

* Don't consider these number as best results, they or on a random 
machine with some effort tuning and core isolation. I saw this hit in 
performance as lower the msg rate on several machine elsewhere and I am 
sure it will reproduce on your tuned machine so you can notice it as well.

Again, this should not block your feature commit but is interesting for 
me to understand and I though someone here might have a good explanation.

thanks,
Alex Rosenbaum
Director R&D Application Acceleration
Mellanox Technologies | Raanana, Israel | +972 (74) 712-9215
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html