netdev - Re: [PATCH net] can: isotp: isotp_rcv

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <24e6da96-a3e5-7b4e-102b-b5676770b80e@hartkopp.net>
Date:   Fri, 28 Jan 2022 08:56:19 +0100
From:   Oliver Hartkopp <socketcan@...tkopp.net>
To:     "Ziyang Xuan (William)" <william.xuanziyang@...wei.com>,
        mkl@...gutronix.de
Cc:     davem@...emloft.net, kuba@...nel.org, linux-can@...r.kernel.org,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH net] can: isotp: isotp_rcv_cf(): fix so->rx race problem

Answering myself ...

I've seen the frame processing sometimes freezes for one second when 
stressing the isotp_rcv() from multiple sources. This finally freezes 
the entire softirq which is either not good and not needed as we only 
need to fix this race for stress tests - and not for real world usage 
that does not create this case.

Therefore I created a V2 patch which uses the spin_trylock() to simply 
drop the incomming frame in the race condition.

https://lore.kernel.org/linux-can/20220128074327.52229-1-socketcan@hartkopp.net/T/

Please take a look, if it also fixes the issue in your test setup.

Thanks & best regards,
Oliver

On 27.01.22 20:44, Oliver Hartkopp wrote:
> Hello Ziyang Xuan,
> 
> On 21.01.22 02:50, Ziyang Xuan (William) wrote:
>>>
>>> On 20.01.22 12:28, Ziyang Xuan (William) wrote:
>>>>>
>>>>> On 20.01.22 07:24, Ziyang Xuan (William) wrote:
>>>>>
>>>>>> I have reproduced the syz problem with Marc's commit, the commit 
>>>>>> can not fix the panic problem.
>>>>>> So I tried to find the root cause for panic and gave my solution.
>>>>>>
>>>>>> Marc's commit just fix the condition that packet size bigger than 
>>>>>> INT_MAX which trigger
>>>>>> tpcon::{idx,len} integer overflow, but the packet size is 4096 in 
>>>>>> the syz problem.
>>>>>>
>>>>>> so->rx.len is 0 after the following logic in isotp_rcv_ff():
>>>>>>
>>>>>> /* get the FF_DL */
>>>>>> so->rx.len = (cf->data[ae] & 0x0F) << 8;
>>>>>> so->rx.len += cf->data[ae + 1];
>>>>>>
>>>>>> so->rx.len is 4096 after the following logic in isotp_rcv_ff():
>>>>>>
>>>>>> /* FF_DL = 0 => get real length from next 4 bytes */
>>>>>> so->rx.len = cf->data[ae + 2] << 24;
>>>>>> so->rx.len += cf->data[ae + 3] << 16;
>>>>>> so->rx.len += cf->data[ae + 4] << 8;
>>>>>> so->rx.len += cf->data[ae + 5];
>>>>>>
>>>>>
>>>>> In these cases the values 0 could be the minimum value in 
>>>>> so->rx.len - but e.g. the value 0 can not show up in isotp_rcv_cf() 
>>>>> as this function requires so->rx.state to be ISOTP_WAIT_DATA.
>>>>
>>>> Consider the scenario that isotp_rcv_cf() and isotp_rcv_cf() are 
>>>> concurrent for the same isotp_sock as following sequence:
>>>
>>> o_O
>>>
>>> Sorry but the receive path is not designed to handle concurrent 
>>> receptions that would run isotp_rcv_cf() and isotp_rcv_ff() 
>>> simultaneously.
>>>
>>>> isotp_rcv_cf()
>>>> if (so->rx.state != ISOTP_WAIT_DATA) [false]
>>>>                          isotp_rcv_ff()
>>>>                          so->rx.state = ISOTP_IDLE
>>>>                          /* get the FF_DL */ [so->rx.len == 0]
>>>> alloc_skb() [so->rx.len == 0]
>>>>                          /* FF_DL = 0 => get real length from next 4 
>>>> bytes */ [so->rx.len == 4096]
>>>> skb_put(nskb, so->rx.len) [so->rx.len == 4096]
>>>> skb_over_panic()
>>>>
>>>
>>> Even though this case is not possible with a real CAN bus due to the 
>>> CAN frame transmission times we could introduce some locking (or 
>>> dropping of concurrent CAN frames) in isotp_rcv() - but this code 
>>> runs in net softirq context ...
>>>
> 
> As discussed off-list I added a spin_lock() in isotp_rcv() as 
> https://www.kernel.org/doc/htmldocs/kernel-locking/lock-softirqs.html 
> suggests.
> 
> Please give this patch[1] a try in your test setup.
> 
> Many thanks,
> Oliver
> 
> [1]: 
> https://lore.kernel.org/linux-can/20220127192429.336335-1-socketcan@hartkopp.net/T/ 
>