netdev - Re: autotuning of send buffer size of a socket

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1210722503.11436.9.camel@w-sridhar2.beaverton.ibm.com>
Date:	Tue, 13 May 2008 16:48:23 -0700
From:	Sridhar Samudrala <sri@...ibm.com>
To:	Shirish Pargaonkar <shirishpargaonkar@...il.com>
Cc:	linux-net@...r.kernel.org, netdev@...r.kernel.org
Subject: Re: autotuning of send buffer size of a socket

On Tue, 2008-05-13 at 08:54 -0500, Shirish Pargaonkar wrote:
> On 5/12/08, Sridhar Samudrala <sri@...ibm.com> wrote:
> > On Mon, 2008-05-12 at 14:00 -0500, Shirish Pargaonkar wrote:
> > > Hello,
> > >
> > > kernel_sendmsg fails with error EAGAIN, yet I no matter how long I try,
> > > I still get the same error and do not see the send buffer size of a socket
> > > changing (increasing)
> > >
> > > The initial buffer sizes are 16384 for send side and 87380 for the receive
> > > side but I see receive side buffer tuning but do not see the same with
> > > send side.
> > >
> > > If tcp does not see a need to increase the send buffer size, wonder why I
> > > get EAGAIN error on this non-blocking socket for kernel_sendmsg!
> >
> > I think the send buffer auto-tuning doesn't happen here because there is
> > already congestion window worth of packets sent that are not yet acknowledged.
> > See tcp_should_expand_sndbuf().
> >
> > Also, the comments for tcp_new_space() says that sndbuf expansion does
> > not work well with largesends. What is the size of your sends?
> >
> > Adding netdev to the CC list.
> >
> > Thanks
> > Sridhar
> >
> > >
> > > I do subscribe to this mailing list so, please send your responses to this
> > > mail address.
> > >
> > > Regards,
> > >
> > > Shirish
> > >
> > > --------------------------------------------------------------------------------------------------
> > > uname -r
> > > 2.6.18-91.el5
> > >
> > >  sysctl -a
> > >
> > > net.ipv4.tcp_rmem = 4096        87380   4194304
> > > net.ipv4.tcp_wmem = 4096        16384   4194304
> > > net.ipv4.tcp_mem = 98304        131072  196608
> > >
> > > net.core.rmem_default = 126976
> > > net.core.wmem_default = 126976
> > > net.core.rmem_max = 131071
> > > net.core.wmem_max = 131071
> > >
> > > net.ipv4.tcp_window_scaling = 1
> > > net.ipv4.tcp_timestamps = 1
> > > net.ipv4.tcp_moderate_rcvbuf = 1
> > >
> > >
> > > cat /proc/sys/net/ipv4/tcp_moderate_rcvbuf
> > > 1
> > >
> > >
> > > CIFS VFS: sndbuf 16384 rcvbuf 87380
> > >
> > > CIFS VFS: sends on sock 0000000009903100, sendbuf 34776, rcvbuf 190080
> > > stuck for 32 seconds,
> > > error: -11
> > > CIFS VFS: sends on sock 0000000009903a00, sndbuf 34776, rcvbuf 138240
> > > stuck for 32 seconds,
> > > error: -11
> > >
> > >
> > > CIFS VFS: sends on sock 0000000009903100, sndbuf 34776, rcvbuf 126720
> > > stuck for 64 seconds,
> > > error: -11
> > >
> > > CIFS VFS: sends on sock 0000000009903100, sndbuf 34776, rcvbuf 222720
> > > stuck for 256 seconds,
> > > error: -11
> > >
> > > I see the socket receive buffer size fluctuating (tcp_moderate_rcvbuf
> > > is 1) but not
> > > the socket send buffer size.
> > > The send buffer size remains fixed, the auto-tuning for send side is
> > > enabled by default,so I do not see it happening here no matter how
> > > long the c ode tries to
> > > kernel_sendmsg after receiving EAGAIN return code.

> Sridhar,
> 
> The size of the sends is 56K.

As David pointed out, the send size may not be an issue. 
When do you see these stalls? Do they happen frequently or only under
stress?

It could be that the receiver is not able to drain the receive queue
causing the send path to be blocked. You could run netstat -tn on
the receiver and take a look at 'Recv-Q' output to see if there is
data pending in the receive queue.

Thanks
Sridhar

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html