netdev - Re: [Regression] stress-ng udp-flood causes kernel panic on Ampere Altra

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YsVmbOqzACeo1rO4@e126311.manchester.arm.com>
Date:   Wed, 6 Jul 2022 11:39:40 +0100
From:   Kajetan Puchalski <kajetan.puchalski@....com>
To:     Will Deacon <will@...nel.org>
Cc:     Florian Westphal <fw@...len.de>,
        Pablo Neira Ayuso <pablo@...filter.org>,
        Jozsef Kadlecsik <kadlec@...filter.org>,
        "David S. Miller" <davem@...emloft.net>,
        Eric Dumazet <edumazet@...gle.com>,
        Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>, Mel Gorman <mgorman@...e.de>,
        lukasz.luba@....com, dietmar.eggemann@....com,
        mark.rutland@....com, broonie@...nel.org,
        netfilter-devel@...r.kernel.org, coreteam@...filter.org,
        netdev@...r.kernel.org, stable@...r.kernel.org,
        regressions@...ts.linux.dev, linux-kernel@...r.kernel.org,
        peterz@...radead.org, kajetan.puchalski@....com
Subject: Re: [Regression] stress-ng udp-flood causes kernel panic on Ampere
 Altra

On Tue, Jul 05, 2022 at 12:24:49PM +0100, Will Deacon wrote:
> > > Sorry, but I have absolutely no context here. We have a handy document
> > > describing the differences between atomic_t and refcount_t:
> > > 
> > > 	Documentation/core-api/refcount-vs-atomic.rst
> > > 
> > > What else do you need to know?
> > 
> > Hmm, and I see a tonne of *_inc_not_zero() conversions in 719774377622
> > ("netfilter: conntrack: convert to refcount_t api") which mean that you
> > no longer have ordering to subsequent reads in the absence of an address
> > dependency.
> 
> I think the patch above needs auditing with the relaxed behaviour in mind,
> but for the specific crash reported here possibly something like the diff
> below?
> 
> Will
> 
> --->8
> 
> diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
> index 082a2fd8d85b..5ad9fcc84269 100644
> --- a/net/netfilter/nf_conntrack_core.c
> +++ b/net/netfilter/nf_conntrack_core.c
> @@ -1394,6 +1394,7 @@ static unsigned int early_drop_list(struct net *net,
>                  * already fired or someone else deleted it. Just drop ref
>                  * and move to next entry.
>                  */
> +               smp_rmb();      /* XXX: Why? */
>                 if (net_eq(nf_ct_net(tmp), net) &&
>                     nf_ct_is_confirmed(tmp) &&
>                     nf_ct_delete(tmp, 0, 0))
> 

Just to follow up, I think you're right, the patch in question should be
audited further for other missing memory barrier issues.
While this one smp_rmb() helps a lot, ie lets the test run for at least
an hour or two, an overnight 6 hour test still resulted in the same
crash somewhere along the way so it looks like it's not the only one
that's needed.