netdev - Re: [PATCH] af_packet: Add Doorbell transmit mode to AF

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20141009160527.GD20940@hmsreliant.think-freely.org>
Date:	Thu, 9 Oct 2014 12:05:27 -0400
From:	Neil Horman <nhorman@...driver.com>
To:	John Fastabend <john.r.fastabend@...el.com>
Cc:	netdev@...r.kernel.org, John Fastabend <john.fastabend@...il.com>,
	Daniel Borkmann <dborkman@...hat.com>,
	Jesper Dangaard Brouer <jbrouer@...hat.com>,
	"John W. Linville" <linville@...driver.com>,
	Florian Westphal <fw@...len.de>, gerlitz.or@...il.com,
	john.ronciak@...el.com, amirv@...lanox.com, eric.dumazet@...il.com,
	danny.zhou@...el.com, Willem de Bruijn <willemb@...gle.com>
Subject: Re: [PATCH] af_packet:  Add Doorbell transmit mode to AF_PACKET
 sockets

On Thu, Oct 09, 2014 at 08:01:07AM -0700, John Fastabend wrote:
> On 10/09/2014 06:36 AM, Neil Horman wrote:
> > This patch adds a variation to the AF_PACKET memory mapped socket transmit
> > mechanism.  Nominally, when using a memory mapped AF_PACKET socket, frames are
> > written into the memory mapped buffer, and then the application calls sendmsg
> > with a NULL buffer which triggers then cleans the mapped space of all pending
> > buffers.
> > 
> > While this provides clean, synchronous operation, improvements can be made.  To
> > this end, I've introduced a doorbell mode of operation to memory mapped packet
> > sockets. When a packet socket is placed into doorbell mode, it write protects
> > the mappings of any process using the packet socket, so that on the first write
> > to it, a kernel trap is generated, which returns the mapping to a read-write
> > state, and forks a task to begin cleaning the buffers on the applications
> > behalf.  This thread contains some hysterisis to continue running a short while
> > after the last buffer has been cleaned, allowing subsquent wrtites to be sent
> > without needing to fork another task.  This allows for additional parallelism in
> > that an application on an smp system can run in parallel with a cleaning task,
> > so that the socket buffer can be filled and emptied in parallel without having
> > to incur multiple system call traps.
> > 
> > I've only done some very rough performance estimates, but early results are
> > promising.  Using this code here:
> > http://wiki.ipxwarzone.com/index.php5?title=Linux_packet_mmap
> > 
> > I made some modifications to support using doorbell mode and compared the time
> > it took to send 1500 packets (each of size 1492 bytes), in basic mmap and
> > doorbell mmaped mode, and used tcpdump to capture the output.  Results:
> > 
> > trace	packets	start time	end time	delta		p/s	size
> > ndb	1500	2.755605	3.000886	0.245281	6115.43	1492b
> > db	1500	4.716448	4.846382	0.129934	11544.32 1492b
> > 
> > Its very rough of course but it would seem I get a 40% increase in throughput
> > when using this method.  I'm sure thats an overestimate, and so more testing is
> > required, but initial results look good.
> > 
> > Signed-off-by: Neil Horman <nhorman@...driver.com>
> > ---
> 
> 
> Thanks Neil, This looks helpful I'll see if I can merge something like this with
> my previous patch. Not likely to have anything by next week though ;)
> 
No worries, I'm on vacation next week anyway :)
Thanks!
Neil

> .John
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html