netdev - Re: [PATCH net-next 7/8] tcp: stronger sk

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANn89iKPVPHQMgMiA=sum_nAjDg6hK0WSzHjP4onUJhYkj1xUQ@mail.gmail.com>
Date: Fri, 19 Dec 2025 09:23:55 +0100
From: Eric Dumazet <edumazet@...gle.com>
To: Christian Ebner <c.ebner@...xmox.com>
Cc: "David S . Miller" <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>, 
	Paolo Abeni <pabeni@...hat.com>, Neal Cardwell <ncardwell@...gle.com>, 
	Simon Horman <horms@...nel.org>, Kuniyuki Iwashima <kuniyu@...gle.com>, 
	Willem de Bruijn <willemb@...gle.com>, netdev@...r.kernel.org, eric.dumazet@...il.com, 
	lkolbe@...iuswillert.com
Subject: Re: [PATCH net-next 7/8] tcp: stronger sk_rcvbuf checks

On Thu, Dec 18, 2025 at 3:58 PM Christian Ebner <c.ebner@...xmox.com> wrote:
>
> On 12/18/25 2:19 PM, Eric Dumazet wrote:
> > On Thu, Dec 18, 2025 at 1:28 PM Christian Ebner <c.ebner@...xmox.com> wrote:
> >>
> >> Hi Eric,
> >>
> >> thank you for your reply!
> >>
> >> On 12/18/25 11:10 AM, Eric Dumazet wrote:
> >>> Can you give us (on receive side) : cat /proc/sys/net/ipv4/tcp_rmem
> >>
> >> Affected users report they have the respective kernels defaults set, so:
> >> - "4096 131072 6291456"  for v.617 builds
> >> - "4096 131072 33554432" with the bumped max value of 32M for v6.18 builds
> >>
> >>> It seems your application is enforcing a small SO_RCVBUF ?
> >>
> >> No, we can exclude that since the output of `ss -tim` show the default
> >> buffer size after connection being established and growing up to the max
> >> value during traffic (backups being performed).
> >>
> >
> > The trace you provided seems to show a very different picture ?
> >
> > [::ffff:10.xx.xx.aa]:8007
> >         [::ffff:10.xx.xx.bb]:55554
> >            skmem:(r0,rb7488,t0,tb332800,f0,w0,o0,bl0,d20) cubic
> > wscale:10,10 rto:201 rtt:0.085/0.015 ato:40 mss:8948 pmtu:9000
> > rcvmss:7168 advmss:8948 cwnd:10 bytes_sent:937478 bytes_acked:937478
> > bytes_received:1295747055 segs_out:301010 segs_in:162410
> > data_segs_out:1035 data_segs_in:161588 send 8.42Gbps lastsnd:3308
> > lastrcv:191 lastack:191 pacing_rate 16.7Gbps delivery_rate 2.74Gbps
> > delivered:1036 app_limited busy:437ms rcv_rtt:207.551 rcv_space:96242
> > rcv_ssthresh:903417 minrtt:0.049 rcv_ooopack:23 snd_wnd:142336 rcv_wnd:7168
> >
> > rb7488 would suggest the application has played with a very small SO_RCVBUF,
> > or some memory allocation constraint (memcg ?)
>
> Thanks for the hint were to look, however we checked that the process is
> not memory constrained and the host has no memory pressure.
>
> Also `strace -f -e socket,setsockopt -p $(pidof proxmox-backup-proxy)`
> shows no syscalls which would change the socket buffer size (though this
> still needs to be double checked by affected users for completeness).
>
> Further, the stalls most often happen mid transfer, starting with the
> expected throughput and even might recover from the stall after some
> time, continue at regular speed again.
>
>
> Status update for v6.18
> -----------------------
>
> In the meantime, a user reported 2 stale connections with running kernel
> 6.18+416dd649f3aa
>
> The tcpdump pattern looks slightly different, here we got repeating
> sequences of:
> ```
> 224     5.407981        10.xx.xx.bb     10.xx.xx.aa     TCP     4162    40068 → 8007 [PSH, ACK]
> Seq=106497 Ack=1 Win=3121 Len=4096 TSval=3198115973 TSecr=3048094015
> 225     5.408064        10.xx.xx.aa     10.xx.xx.bb     TCP     66      8007 → 40068 [ACK] Seq=1
> Ack=110593 Win=4 Len=0 TSval=3048094223 TSecr=3198115973
> ```
>
> The perf trace for `tcp:tcp_rcvbuf_grow` came back empty while in stale
> state, tracing with:
> ```
> perf record -a -e tcp:tcp_rcv_space_adjust,tcp:tcp_rcvbuf_grow
> perf script
> ```
> produced some output as shown below, so it seems that tcp_rcvbuf_grow()
> is never called in that case, while tcp_rcv_space_adjust() is.

Autotuning is not enabled for your case, somehow the application is
not behaving as expected,
so maybe you have to change tcp_rmem[2] if a driver is allocating
order-2 pages for the 9K frames.

You have not given what  was on the sender side (linux or other stack ?)