netdev - FlameGraph of mlx4 early drop with order-0 pages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160415214034.6ffae9ee@redhat.com>
Date:	Fri, 15 Apr 2016 21:40:34 +0200
From:	Jesper Dangaard Brouer <brouer@...hat.com>
To:	Mel Gorman <mgorman@...hsingularity.net>,
	linux-mm <linux-mm@...ck.org>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	Brenden Blanco <bblanco@...mgrid.com>
Cc:	brouer@...hat.com, tom@...bertland.com,
	alexei.starovoitov@...il.com, ogerlitz@...lanox.com,
	daniel@...earbox.net, brouer@...hat.com, eric.dumazet@...il.com,
	ecree@...arflare.com, john.fastabend@...il.com, tgraf@...g.ch,
	johannes@...solutions.net, eranlinuxmellanox@...il.com
Subject: FlameGraph of mlx4 early drop with order-0 pages

Hi Mel,

I did an experiment that you might find interesting.  Using Brenden's
early drop with eBPF in the mxl4 driver.  I changed the mlx4 driver to
use order-0 pages.  It usually use order-3 pages to amortize the cost
of calling the page allocator (which is problematic for other reasons,
like memory pin-down, latency spikes and multi CPU scalability)

With this change I could do around 12Mpps (Mill packet per sec) drops,
usually does 14.5Mpps (limited due to a HW setup/limit, with idle cycles). 

Looking at the perf report as a FlameGraph, the page allocator clearly
show up as the bottleneck: 

http://people.netfilter.org/hawk/FlameGraph/flamegraph-mlx4-order0-pages-eBPF-XDP-drop.svg

Signing off, heading for the plane soon... see you at MM-summit!
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer