netdev - Re: kernel BUG at net/core/skbuff.c:4219

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Wed, 19 Oct 2022 12:10:30 -0700
From:   Tomas Hruby <tomas@...era.io>
To:     eric.dumazet@...il.com
Cc:     edumazet@...gle.com, herbert@...dor.apana.org.au,
        jpiotrowski@...ux.microsoft.com, linux-kernel@...r.kernel.org,
        netdev@...r.kernel.org, seh@...ix.com
Subject: Re: kernel BUG at net/core/skbuff.c:4219

> > Hi,
> >
> > One of our Flatcar users has been hitting the kernel BUG in the subject line
> > for the past year (https://github.com/flatcar/Flatcar/issues/378). This was
> > first reported when on 5.10.25, but has been happening across kernel updates,
> > most recently with 5.15.63. The nodes where this happens are AWS EC2 instances,
> > using ENA and calico networking in eBPF mode with VXLAN encapsulation. When
> > GRO/GSO is enabled, the host hits this bug and prints the following stacktrace:
>
>
> I suspect eBPF code lowers gso_size ?
>
> gso stack is not able to arbitrarily segment a GRO packet after gso_size
> being changed.

Based on the stack trace, it happens for a tcp packet. Since it seems
like it is on egress from a host, I suspect that it is encapsulating a
tcp packet into a vxlan tunnel and useds bpf_skb_adjust_room() here
https://github.com/projectcalico/calico/blob/master/felix/bpf-gpl/nat.h#L77-L80
Not sure if it should use BPF_F_ADJ_ROOM_FIXED_GSO in that case.

Calico uses the flag when decapsulating packets here
https://github.com/projectcalico/calico/blob/master/felix/bpf-gpl/nat.h#L143
and it uses the flag as the packet is UDP.

Any of that could cause the BUG?

Tomas