netdev - Re: can TCP socket send buffer be over used?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 4 Aug 2010 02:00:13 -0600
From:	Jack Zhang <jack.zhang2011@...il.com>
To:	Bill Fink <billfink@...dspring.com>
Cc:	netdev@...r.kernel.org
Subject: Re: can TCP socket send buffer be over used?

Hi Bill,

Thanks a lot for your help.

It does make sense!

As I'm writing this part into my master thesis, do you happen to know
which part in the source code I can maybe use as a proof in the thesis
that Linux uses 1/4 of the doubled buffer size for metadata?

Thanks,
Jack

On 4 August 2010 01:20, Bill Fink <billfink@...dspring.com> wrote:
> On Tue, 3 Aug 2010, Jack Zhang wrote:
>
>> Hi there,
>>
>> I'm doing experiments with (modified*) software iSCSI over a link with
>> an emulated Round-Trip Time (RTT) of 100 ms by netem.
>>
>> For example, when I set the send buffer size to 128 KB, i could get a
>> throughput up to 43 Mbps, which seems to be impossible as the (buffer
>> size) / RTT is only 10 Mbps.
>
> I'm not sure what's going on with this first case.
>
>> And When I set the send buffer size to 512 KB, i can get a throughput
>> up to 60 Mbps, which also seems to be impossible as the (buffer size)
>> / RTT is only 40 Mbps.
>
> But this case seems just about right.  Linux doubles the requested
> buffer size, then uses one quarter of that for overhead (not half),
> so you effectively get 50% more than requested (2X * 3/4 = 1.5X).
> Plugging your case into bc:
>
> wizin% bc
> scale=10
> 512*1024*8/0.100/10^6*3/2
> 62.9145600000
>
>                                                -Bill
>
>
>
>> I understand that when the buffer size is set to 128 KB, I actually
>> got a buffer of 256 KB as the kernel doubles the buffer size. I also
>> understand that half the doubled buffer size is used for meta data
>> instead of the actual data to be transferred. So basically the
>> effective buffer sizes for the two examples  are just 128 KB and 512
>> KB respectively.
>>
>> So I was confused because, theoretically, send buffers of 128 KB and
>> 512 KB should achieve no more than 10 Mbps and 40 Mbps respectively
>> but I was able to get way much more than the theoretical limit. So
>> I was wondering is there any chance the send buffer can be "overused"?
>> or there is some other mechanism inside TCP is doing some
>> optimization?
>>
>> * the modification is to disable "TCP_NODELAY" , enable
>> "use_clustering" for SCSI, and set different send buffer sizes for the
>> TCP socket buffer.
>>
>> Any idea will be highly appreciated.
>>
>> Thanks a lot!
>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html