netdev - Re: [PATCH net-next] tcp: update window_clamp together with scaling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240409215930.2f27laoh4uoj6s7x@lgud-hli>
Date: Tue, 9 Apr 2024 14:59:30 -0700
From: Hechao Li <hli@...flix.com>
To: Neal Cardwell <ncardwell@...gle.com>
Cc: Eric Dumazet <edumazet@...gle.com>,
	"David S . Miller" <davem@...emloft.net>,
	Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>,
	Soheil Hassas Yeganeh <soheil@...gle.com>, netdev@...r.kernel.org,
	kernel-developers@...flix.com, Tycho Andersen <tycho@...ho.pizza>
Subject: Re: [PATCH net-next] tcp: update window_clamp together with
 scaling_ratio

On 24/04/09 12:51PM, Neal Cardwell wrote:
> On Wed, Apr 3, 2024 at 12:30 PM Hechao Li <hli@...flix.com> wrote:
> > The application is kafka and it has a default config of 64KB SO_RCVBUF
> > (https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html#receive-buffer-bytes)
> > so in this case it's limitted by SO_RCVBUF and not tcp_rmem. It also has
> > a default request timeout 30 seconds
> > (https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html#request-timeout-ms)
> > The combination of these two configs requires the certain amount of app
> > data (in our case 10M) to be transfer within 30 seconds. But a 32k
> > window size can't achieve this, causing app timeout. Our goal was to
> > upgrade the kernel without having to update applications if possible.
> 
> Hechao, can you please file a bug against Kafka to get them to stop
> using SO_RCVBUF, and allow receive buffer autotuning? This default
> value of 64 Kbytes will cripple performance in many scenarios,
> especially for WAN traffic.
> 
> I guess that would boil down to asking for the default
> receive.buffer.bytes to be -1 rather than 64 Kbytes.
> 
> Looks like you can file bugs here:
>   https://issues.apache.org/jira/browse/KAFKA
> 
> thanks,
> neal

Makes sense. Filed: https://issues.apache.org/jira/browse/KAFKA-16496

I don't know the reason why kafka set 64k as the default in the first
place. But I was wondering if there is any use case that requires the
user to set SO_RCVBUF rather than auto tuning? Eventually do we want to
depreacte the support of setting SO_RCVBUF at all?

Thank you.

Hechao