netdev - Re: [PATCH net] net: enforce xmit_recursion for devices with a queue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <dea7ebea-075d-ea18-9d38-f96202929dd1@gmail.com>
Date:   Thu, 14 Mar 2019 10:51:49 -0700
From:   Eric Dumazet <eric.dumazet@...il.com>
To:     Sabrina Dubroca <sd@...asysnail.net>,
        Eric Dumazet <eric.dumazet@...il.com>
Cc:     netdev@...r.kernel.org, Jianlin Shi <jishi@...hat.com>,
        Stefano Brivio <sbrivio@...hat.com>
Subject: Re: [PATCH net] net: enforce xmit_recursion for devices with a queue



On 03/14/2019 10:40 AM, Sabrina Dubroca wrote:
> 2019-03-14, 07:56:10 -0700, Eric Dumazet wrote:
>>
>>
>> On 03/14/2019 07:15 AM, Sabrina Dubroca wrote:
>>> 2019-03-14, 05:58:03 -0700, Eric Dumazet wrote:
>>>>
>>>>
>>>> On 03/14/2019 03:15 AM, Sabrina Dubroca wrote:
>>>>> Commit 745e20f1b626 ("net: add a recursion limit in xmit path")
>>>>> introduced a recursion limit, but it only applies to devices without a
>>>>> queue. Virtual devices with a queue (either because they don't have
>>>>> the IFF_NO_QUEUE flag, or because the administrator added one) can
>>>>> still cause an unbounded recursion, via __dev_queue_xmit ->
>>>>> __dev_xmit_skb -> qdisc_run -> __qdisc_run -> qdisc_restart ->
>>>>> sch_direct_xmit -> dev_hard_start_xmit . Jianlin reported this in a
>>>>> setup with 16 gretap devices stacked on top of one another.
>>>>>
>>>>> This patch prevents the stack overflow by incrementing xmit_recursion in
>>>>> code paths that can call dev_hard_start_xmit() (like commit 745e20f1b626
>>>>> did). If the recursion limit is exceeded, the packet is enqueued and the
>>>>> qdisc is scheduled.
>>>>>
>>>>> Reported-by: Jianlin Shi <jishi@...hat.com>
>>>>> Signed-off-by: Sabrina Dubroca <sd@...asysnail.net>
>>>>> Reviewed-by: Stefano Brivio <sbrivio@...hat.com>
>>>>
>>>> Hi Sabrina, thanks for the patch.
>>>>
>>>> Can't we detect this in the control path instead ?
>>>
>>> I don't see how. You could have a perfectly reasonable set of gretap
>>> devices that trigger this situation from simply reshuffling the IP
>>> addresses:
>>>
>>> gretap$x remote 1.1.$((x-1)).{1,2}
>>> (all those addresses set on a single veth device)
>>>
>>> Then you move those addresses to the corresponding device
>>> (1.1.${x}.{1,2} on gretap$x), and your machine crashes.
>>>
>>
>> If this only can be done with gretap, why gretap cant implement the protection,
>> outside of the fast path ?
> 
> It's not just gretap. VXLAN will do the same as long as you add a
> qdisc. I expect other types of tunnels to behave like that.
> 

It might make sense to add a helper using dev_queue_xmit()
for tunnel users.

Then remove the xmit recursion stuff out of the dev_queue_xmit()

Lets make the fast path fast again.