netdev - Re: 6.1: possible bug with netfilter conntrack?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230112233808.GA19463@breakpoint.cc>
Date:   Fri, 13 Jan 2023 00:38:08 +0100
From:   Florian Westphal <fw@...len.de>
To:     "Russell King (Oracle)" <linux@...linux.org.uk>
Cc:     netdev@...r.kernel.org, netfilter-devel@...r.kernel.org,
        coreteam@...filter.org
Subject: Re: 6.1: possible bug with netfilter conntrack?

Russell King (Oracle) <linux@...linux.org.uk> wrote:
> Hi,
> 
> I've noticed that my network at home is rather struggling, and having
> done some investigation, I find that the router VM is dropping packets
> due to lots of:
> 
> nf_conntrack: nf_conntrack: table full, dropping packet
> 
> I find that there are about 2380 established and assured connections
> with a destination of my incoming mail server with destination port 25,
> and 2 packets. In the reverse direction, apparently only one packet was
> sent according to conntrack. E.g.:
> 
> tcp      6 340593 ESTABLISHED src=180.173.2.183 dst=78.32.30.218
> sport=49694 dport=25 packets=2 bytes=92 src=78.32.30.218
> dst=180.173.2.183 sport=25 dport=49694 packets=1 bytes=44 [ASSURED]
> use=1

Non-early-evictable entry that will expire in ~4 days, so not really
surprising that this eventually fills the table.

I'd suggest to reduce the
net.netfilter.nf_conntrack_tcp_timeout_established
sysctl to something more sane, e.g. 2 minutes or so unless you need
to have longer timeouts.

But this did not change, so not the root cause of this problem.

> However, if I look at the incoming mail server, its kernel believes
> there are no incoming port 25 connetions, which matches exim.
> 
> I hadn't noticed any issues prior to upgrading from 5.16 to 6.1 on the
> router VM, and the firewall rules have been the same for much of
> 2021/2022.
>
> Is this is known issue? Something changed between 5.16 and 6.1 in the
> way conntrack works?

Nothing that should have such an impact.

Does 'sysctl net.netfilter.nf_conntrack_tcp_loose=0' avoid the buildup
of such entries? I'm wondering if conntrack misses the connection
shutdown or if its perhaps triggering the entries because of late
packets or similar.

If that doesn't help. you could also check if

'sysctl net.netfilter.nf_conntrack_tcp_be_liberal=1' helps -- if it
does, its time for more debugging but its too early to start digging
atm.  This would point at conntrack ignoring/discarding fin/reset
packets.