netdev - Soft lockup in tc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <7394f89e-e8a5-5fb2-ee04-63bf1c4ef6e7@mellanox.com>
Date:   Mon, 12 Dec 2016 11:43:08 +0200
From:   Shahar Klein <shahark@...lanox.com>
To:     <netdev@...r.kernel.org>
CC:     Roi Dayan <roid@...lanox.com>, David Miller <davem@...emloft.net>,
        "Cong Wang" <xiyou.wangcong@...il.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        "Jiri Pirko" <jiri@...lanox.com>,
        John Fastabend <john.fastabend@...il.com>,
        "Or Gerlitz" <ogerlitz@...lanox.com>,
        Hadar Hen Zion <hadarh@...lanox.com>
Subject: Soft lockup in tc_classify

Hi All,

sorry for the spam, the first time was sent with html part and was rejected.

We observed an issue where a classifier instance next member is pointing 
back to itself, causing a CPU soft lockup.
We found it by running traffic on many udp connections and then adding a 
new flower rule using tc.

We added a quick workaround to verify it:

In tc_classify:

         for (; tp; tp = rcu_dereference_bh(tp->next)) {
                 int err;
+               if (tp == tp->next)
+                     RCU_INIT_POINTER(tp->next, NULL);

We also had a print here showing tp->next is pointing to tp. With this 
workaround we are not hitting the issue anymore.
We are not sure we fully understand the mechanism here - with the rtnl 
and rcu locks.
We'll appreciate your help solving this issue.

Thanks,
Shahar