netdev - Re: STMMAC driver with TSO enabled issue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <c1c9025e-f75b-9ae1-4513-24615308e0af@nvidia.com>
Date:   Thu, 24 May 2018 11:28:48 +0530
From:   Bhadram Varka <vbhadram@...dia.com>
To:     Jose Abreu <Jose.Abreu@...opsys.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        Joao Pinto <Joao.Pinto@...opsys.com>
Subject: Re: STMMAC driver with TSO enabled issue

Hi Jose,

On 5/17/2018 7:43 PM, Jose Abreu wrote:
> Hi Bhadram,
> 
> On 15-05-2018 07:44, Bhadram Varka wrote:
>> Hi Jose,
>>
>> On 5/10/2018 9:15 PM, Jose Abreu wrote:
>>>
>>>
>>> On 10-05-2018 16:08, Bhadram Varka wrote:
>>>> Hi Jose,
>>>>
>>>> On 5/10/2018 7:59 PM, Jose Abreu wrote:
>>>>> Hi Bhadram,
>>>>>
>>>>> On 10-05-2018 09:55, Jose Abreu wrote:
>>>>>> ++net-dev
>>>>>>
>>>>>> Hi Bhadram,
>>>>>>
>>>>>> On 09-05-2018 12:03, Bhadram Varka wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Thanks for responding.
>>>>>>>
>>>>>>> Tried below suggested way. Still observing the issue -
>>>>>> It seems stmmac has a bug in the RX side when using TSO
>>>>>> which is
>>>>>> causing all the RX descriptors to be consumed. The stmmac_rx()
>>>>>> function will need to be refactored. I will send a fix ASAP.
>>>>>
>>>>> Are you using this patch [1] ? Because there is a problem with
>>>>> the patch. By adding the previously removed call to
>>>>> stmmac_init_rx_desc() TSO works okay in my setup.
>>>>>
>>>>
>>>> No. I don't have this change in my code base. I am using
>>>> net-next tree.
>>>>
>>>> Can you please post the change for which TSO works ? I can help
>>>> you with the testing.
>>>
>>> It should work with net-next because patch was not merged yet ...
>>> Please send me the output of "dmesg | grep -i stmmac", since boot
>>> and your full register values (from 0x0 to 0x12E4).
>>>
>>
>> [root@...rm ~]# dmesg | grep -i dwc
>> [    6.925005] dwc-eth-dwmac 2490000.ethernet: Cannot get CSR
>> clock
>> [    6.933657] dwc-eth-dwmac 2490000.ethernet: no reset control
>> found
>> [    6.955325] dwc-eth-dwmac 2490000.ethernet: User ID: 0x10,
>> Synopsys ID: 0x41
>> [    6.962379] dwc-eth-dwmac 2490000.ethernet:  DWMAC4/5
>> [    6.967434] dwc-eth-dwmac 2490000.ethernet: DMA HW
>> capability register supported
>> [    6.974827] dwc-eth-dwmac 2490000.ethernet: RX Checksum
>> Offload Engine supported
>> [    6.982915] dwc-eth-dwmac 2490000.ethernet: TX Checksum
>> insertion supported
>> [    6.991235] dwc-eth-dwmac 2490000.ethernet: Wake-Up On Lan
>> supported
>> [    6.998974] dwc-eth-dwmac 2490000.ethernet: TSO supported
>> [    7.006422] dwc-eth-dwmac 2490000.ethernet: TSO feature enabled
>> [    7.012581] dwc-eth-dwmac 2490000.ethernet: Enable RX
>> Mitigation via HW Watchdog Timer
>> [    7.236391] dwc-eth-dwmac 2490000.ethernet eth0: device MAC
>> address 4a:d1:e3:58:cb:7a
>> [    7.333414] dwc-eth-dwmac 2490000.ethernet eth0: IEEE
>> 1588-2008 Advanced Timestamp supported
>> [    7.342441] dwc-eth-dwmac 2490000.ethernet eth0: registered
>> PTP clock
>> [   10.157066] dwc-eth-dwmac 2490000.ethernet eth0: Link is Up
>> - 1Gbps/Full - flow control off
>> [root@...rm ~]# dmesg | grep -i stmma
>> [    7.020567] libphy: stmmac: probed
>> [    7.316295] Broadcom BCM89610 stmmac-0:00: attached PHY
>> driver [Broadcom BCM89610] (mii_bus:phy_addr=stmmac-0:00, irq=64)
>>
>> I will get the register details -
>>
>> FYI - TSO works fine with single channel. I see the issue only
>> if multi channel enabled (supports 4 Tx/Rx channels).
>>
> 
> And normal data transfer works okay with multi channel, right? I
> will need the register details to proceed ... You could also try
> git bisect ...
> 

Yes - normal data transfers works fine. Issue observed only driver gets 
TSO packet. Looks like TX DMA channel hang.

After adding few debug logs - observed that while processing second or 
third descriptor TX DMA hangs.

[85788.137498] stmmac_tso_xmit: tcphdrlen 32, hdr_len 66, pay_len 1392, 
mss 1448
[85788.144634]  skb->len 7306, skb->data_len 5848
[..]
[85788.274876] 025 [0x82795190]: 0x0 0x0 0x5a8 0xc4000000
[85788.280020] 026 [0x827951a0]: 0xf854e000 0xf854e042 0x5700042 0xa0441c48
[85788.286730] 027 [0x827951b0]: 0xf854f000 0x0 0x16d8 0x90000000
[...]


After some time if check Tx descriptor status - then I see only below

[..]
[85788.286730] 027 [0x827951b0]: 0xf854f000 0x0 0x16d8 0x90000000

index 025 and 026 descriptors processed but not index 027.

At this stage Tx DMA is always in below state -

■ 3'b011: Running (Reading Data from system memory
buffer and queuing it to the Tx buffer (Tx FIFO))

Thanks,
Bhadram.