netdev - Re: tcp: kernel BUG at net/core/skbuff.c:3574!

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CALrw=nG5-Qyi8f0j6-dmkVts4viX24j755gEiUNTQDoXzXv1XQ@mail.gmail.com>
Date:   Tue, 21 Dec 2021 17:16:04 +0000
From:   Ignat Korchagin <ignat@...udflare.com>
To:     Paolo Abeni <pabeni@...hat.com>, Eric Dumazet <edumazet@...gle.com>
Cc:     netdev <netdev@...r.kernel.org>,
        "David S. Miller" <davem@...emloft.net>,
        David Ahern <dsahern@...nel.org>,
        Jakub Kicinski <kuba@...nel.org>,
        kernel-team <kernel-team@...udflare.com>
Subject: Re: tcp: kernel BUG at net/core/skbuff.c:3574!

On Tue, Dec 21, 2021 at 3:40 PM Paolo Abeni <pabeni@...hat.com> wrote:
>
> On Tue, 2021-12-21 at 06:16 -0800, Eric Dumazet wrote:
> > On Tue, Dec 21, 2021 at 4:19 AM Ignat Korchagin <ignat@...udflare.com> wrote:
> > >
> > > Hi netdev,
> > >
> > > While trying to reproduce a different rare bug we're seeing in
> > > production I've triggered below on 5.15.9 kernel and confirmed on the
> > > latest netdev master tree:
> > >
> >
> > Nothing comes to mind. skb_shift() has not been recently changed.
> >
> > Why are you disabling TSO exactly ?
> >
> > Is GRO being used on veth needed to trigger the bug ?
> > (GRO was added recently to veth, I confess I did not review the patches)

Yes, it seems enabling GRO for veth actually enables NAPI codepaths,
which trigger this bug (and actually another one we're investigating).
Through trial-and-error it seems disabling TSO is more likely to
trigger it at least in my dev environment. I'm not sure if this bug is
somehow related to the other one we're investigating, but once we have
a fix here I can try to verify before posting it to the mailing list.

> This is very likely my fault. I'm investigating it right now.

Thank you very much! Let me know if I can help somehow.

Ignat

> Thanks for the head-up.
>
> Paolo
>