netdev - Re: [Patch net 00/16] net_sched: fix races with RCU callbacks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMDBHYJiQWSW490syWS195vZ3iYLaczPtNt8aYdqska5q+YgHQ@mail.gmail.com>
Date:   Tue, 31 Oct 2017 15:13:29 -0400
From:   Lucas Bates <lucasb@...atatu.com>
To:     Jamal Hadi Salim <jhs@...atatu.com>
Cc:     Cong Wang <xiyou.wangcong@...il.com>,
        Linux Kernel Network Developers <netdev@...r.kernel.org>,
        Chris Mi <chrism@...lanox.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Jiri Pirko <jiri@...nulli.us>,
        John Fastabend <john.fastabend@...il.com>,
        "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Subject: Re: [Patch net 00/16] net_sched: fix races with RCU callbacks

On Tue, Oct 31, 2017 at 2:55 PM, Lucas Bates <lucasb@...atatu.com> wrote:
> On Tue, Oct 31, 2017 at 7:00 AM, Jamal Hadi Salim <jhs@...atatu.com> wrote:
>> On 17-10-31 01:44 AM, Cong Wang wrote:
>>>
>>> Can you try this patch? From your stack trace it is not clear where
>>> the cause is, but we know that the crash is in __tcf_idr_release(),
>>> this is how I came up with the following patch:
>>>
>>>
>>> diff --git a/include/net/act_api.h b/include/net/act_api.h
>>> index b944e0eb93be..5072446d5f06 100644
>>> --- a/include/net/act_api.h
>>> +++ b/include/net/act_api.h
>>> @@ -122,7 +122,9 @@ void tcf_idrinfo_destroy(const struct tc_action_ops
>>> *ops,
>>>
>>>   static inline void tc_action_net_exit(struct tc_action_net *tn)
>>>   {
>>> +       rtnl_lock();
>>>          tcf_idrinfo_destroy(tn->ops, tn->idrinfo);
>>> +       rtnl_unlock();
>>>          kfree(tn->idrinfo);
>>>   }
>>>
>>>
>>
>> Looks like an excellent bet;-> tdc kills the container at the end of the
>> test.
>
> Unfortunately it doesn't seem to have had any effect, I'm still seeing
> the same bug as yesterday. At Jamal's suggestion I put in a delay
> after tdc completed running the tests but before it deleted the
> container - and I didn't run into the bug after many runs. I had no
> luck getting serial console access on any of our other systems so I
> have another stack trace photo which I'll send to you directly.

I forgot to mention: it appears the bug appears to be related to
deleting the container. Test d052 is the last to run before tdc does
its teardown and deletes the container; if I add the sleep the bug
never gets triggered.