linux-kernel - RE: [PATCH 1/1] Drivers: net-next: hyperv: Increase the size of the sendbuf region

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 1 Aug 2014 17:51:44 +0000
From:	KY Srinivasan <kys@...rosoft.com>
To:	David Miller <davem@...emloft.net>
CC:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"devel@...uxdriverproject.org" <devel@...uxdriverproject.org>,
	"olaf@...fle.de" <olaf@...fle.de>,
	"apw@...onical.com" <apw@...onical.com>,
	"jasowang@...hat.com" <jasowang@...hat.com>
Subject: RE: [PATCH 1/1] Drivers: net-next: hyperv: Increase the size of the
 sendbuf region

> -----Original Message-----
> From: David Miller [mailto:davem@...emloft.net]
> Sent: Thursday, July 31, 2014 9:59 PM
> To: KY Srinivasan
> Cc: netdev@...r.kernel.org; linux-kernel@...r.kernel.org;
> devel@...uxdriverproject.org; olaf@...fle.de; apw@...onical.com;
> jasowang@...hat.com
> Subject: Re: [PATCH 1/1] Drivers: net-next: hyperv: Increase the size of the
> sendbuf region
> 
> From: "K. Y. Srinivasan" <kys@...rosoft.com>
> Date: Wed, 30 Jul 2014 18:35:49 -0700
> 
> > For forwarding scenarios, it will be useful to allocate larger
> > sendbuf. Make the necessary adjustments to permit this.
> >
> > Signed-off-by: K. Y. Srinivasan <kys@...rosoft.com>
> 
> This needs more information.
> 
> You're increasing the size by 16 times, 1MB --> 16MB, thus less cache locality.
> 
> You're also now using vmalloc() memory, thus more TLB misses and
> thrashing.
> 
> This must have a negative impact on performance, and you have to test for
> that and quantify it when making a change as serious as this one.
> 
> You also haven't gone into detail as to why forwarding scenerios require
> more buffer space, than say thousands of local sockets sending bulk TCP
> data.

David,

Intel did some benchmarking on our network throughput when Linux on Hyper-V was used as a gateway.
This fix gave us almost a 1 Gbps additional throughput on about 5Gbps base throughput we had prior to
Increasing the sendbuf size. The sendbuf mechanism is a copy based transport that we have which is clearly
more optimal than the copy-free page flipping mechanism (for small packets). In the forwarding scenario, we deal
only with MTU sized packets, and increasing the size of the senbuf area gave us the additional performance.

For what it is worth, Windows guests on Hyper-V, I am told use similar sendbuf size as well.

The exact value of sendbuf I think is less important than the fact that it needs to be larger than what Linux can
allocate as physically contiguous memory. Thus the change over to allocating via vmalloc(). As you know we currently
allocate 16MB receive buffer and we use vmalloc there for allocation. Also the low level channel code has already been
modified to deal with physically dis-contiguous memory in the ringbuffer setup.

Again based on experimentation Intel did, they say there was some improvement in throughput as the sendbuf size was
Increased up to 16MB and there was no effect on throughput beyond 16MB. Thus I chose 16MB here.

Increasing the sendbuf value makes a material difference in small packet handling. Let me know what I should do to
make this patch acceptable.

Regards,

K. Y

> 
> I'm not applying this, it needs a lot more work.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/