netdev - Re: [PATCH net-next] tcp: update window_clamp together with scaling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANn89iJ3YrSg-Y+g65vowMtBzvNokT2N7ffk4=uw33k3SsePPA@mail.gmail.com>
Date: Wed, 3 Apr 2024 18:43:01 +0200
From: Eric Dumazet <edumazet@...gle.com>
To: Hechao Li <hli@...flix.com>
Cc: "David S . Miller" <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>, 
	Paolo Abeni <pabeni@...hat.com>, Soheil Hassas Yeganeh <soheil@...gle.com>, netdev@...r.kernel.org, 
	kernel-developers@...flix.com, Tycho Andersen <tycho@...ho.pizza>
Subject: Re: [PATCH net-next] tcp: update window_clamp together with scaling_ratio

On Wed, Apr 3, 2024 at 6:30 PM Hechao Li <hli@...flix.com> wrote:
>
> On 24/04/03 04:49PM, Eric Dumazet wrote:
> > On Wed, Apr 3, 2024 at 4:22 PM Eric Dumazet <edumazet@...gle.com> wrote:
> > >
> > > On Tue, Apr 2, 2024 at 11:56 PM Hechao Li <hli@...flix.com> wrote:
> > > >
> > > > After commit dfa2f0483360 ("tcp: get rid of sysctl_tcp_adv_win_scale"),
> > > > we noticed an application-level timeout due to reduced throughput. This
> > > > can be reproduced by the following minimal client and server program.
> > > >
> > > > server:
> > > >
> > > ...
> > > >
> > > > Before the commit, it takes around 22 seconds to transfer 10M data.
> > > > After the commit, it takes 40 seconds. Because our application has a
> > > > 30-second timeout, this regression broke the application.
> > > >
> > > > The reason that it takes longer to transfer data is that
> > > > tp->scaling_ratio is initialized to a value that results in ~0.25 of
> > > > rcvbuf. In our case, SO_RCVBUF is set to 65536 by the application, which
> > > > translates to 2 * 65536 = 131,072 bytes in rcvbuf and hence a ~28k
> > > > initial receive window.
> > >
> > > What driver are you using, what MTU is set ?
>
> The driver is AWS ENA driver. This is cross-region/internet traffic, so
> the MTU is 1500.
>
> > >
> > > If you get a 0.25 ratio, that is because a driver is oversizing rx skbs.
> > >
> > > SO_RCVBUF 65536 would map indeed to 32768 bytes of payload.
> > >
>
> The 0.25 ratio is the initial default ratio calculated using
>
> #define TCP_DEFAULT_SCALING_RATIO ((1200 << TCP_RMEM_TO_WIN_SCALE) / \
>                                    SKB_TRUESIZE(4096))
>
> I think this is a constant 0.25, no?

This depends on skb metadata size, which changes over time.

With MAX_SKB_FRAGS == 17, this is .25390625

With MAX_SKB_FRAGS == 45, this is .234375


>
> Later with skb->len/skb->truesize, we get 0.66. However, the window
> can't grow to this ratio because window_clamp stays at the initial
> value, which is the initial tcp_full_space(sk), which is roughly 0.25 *
> rcvbuf.

Sure. Please address Jakub feedback about tests.