netdev - Re: [RFC PATCH net-next 2/8] sfc: batch up RX delivery on EF10

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1461086440.10638.208.camel@edumazet-glaptop3.roam.corp.google.com>
Date:	Tue, 19 Apr 2016 10:20:40 -0700
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Edward Cree <ecree@...arflare.com>
Cc:	netdev@...r.kernel.org, David Miller <davem@...emloft.net>,
	Jesper Dangaard Brouer <brouer@...hat.com>,
	linux-net-drivers@...arflare.com
Subject: Re: [RFC PATCH net-next 2/8] sfc: batch up RX delivery on EF10

On Tue, 2016-04-19 at 17:36 +0100, Edward Cree wrote:

> > We have to really invent something better, like a real pipeline, instead
> > of hacks like this, adding complexity everywhere.
> I'm not sure what you mean by 'a real pipeline' in this context, could you
> elaborate?
> 
> > Have you tested this on cpus with tiny caches, like 32KB ?
> I haven't.  Is the concern here that the first packet's headers (we read 128
> bytes into the linear area) and/or skb will get pushed out of the dcache as
> we process further packets?
> 
> At least for sfc, it's highly unlikely that these cards will be used in low-
> powered systems.  For the more general case, I suppose the answer would be a
> tunable to set the maximum length of the RX list to less than the NAPI budget.
> Fundamentally this kind of batching is trading dcache usage for icache usage.
> 
> 
> Incidentally, this patch is very similar to what Jesper proposed for mlx5 in
> an RFC back in February: http://article.gmane.org/gmane.linux.network/397379
> So I'm a little surprised this bit is controversial, though I'm not surprised
> the rest of the series is ;)

It seems all the discussions about fast kernel networking these days is
adding yet another queues, code duplication and complexity, batches, and
add latencies, on top of a single NIC RX queue.

Apparently the multiqueue nature of a NIC is obsolete and people want to
process 10+Mpps on a single queue.

But in the end, we do not address fundamental issues, like number of
cache line misses per incoming TCP packet, and per outgoing TCP packet.
(Or UDP if that matters).
Number of atomic ops to synchronize all accesses to common resources.
(memory, queue limits, queues)

Also issues with dealing with all the queues (percpu backlog in
net/core/dev.c, socket backlog, prequeue for TCP, ...)

We could probably get ~100% improvement in UDP if we really cared, just
by changing net/ipv[46]/udp.c, not changing other layers.