netdev - Re: [BUG Report] hns3: tx_timeout on high memory pressure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <F90EE18D-1B5D-4FB2-ADEB-EF02A2922B7F@gmail.com>
Date: Mon, 23 Sep 2024 21:11:42 +0800
From: Miao Wang <shankerwangmiao@...il.com>
To: Jijie Shao <shaojijie@...wei.com>
Cc: netdev@...r.kernel.org,
 陈晟祺 <harry-chen@...look.com>,
 张宇翔 <zz593141477@...il.com>,
 陈嘉杰 <jiegec@...com>,
 Mirror Admin Tuna <mirroradmin@...a.tsinghua.edu.cn>,
 Salil Mehta <salil.mehta@...wei.com>,
 Yisen Zhuang <yisen.zhuang@...wei.com>
Subject: Re: [BUG Report] hns3: tx_timeout on high memory pressure



> 2024年9月23日 20:58，Jijie Shao <shaojijie@...wei.com> 写道：
> 
> 
> on 2024/9/23 0:38, Miao Wang wrote:
>> It seems that hns3 driver is trying to allocating 16 continuous pages of memory
>> when initializing, which could fail when the system is under high memory
>> pressure.
>> 
>> I have two questions about this:
>> 
>> 1. Is it expected that tx timeout really related to the high memory pressure,
>>    or the driver does not work properly under such condition?
>> 
>> 2. Can allocating continuous pages of memory on initialization can be avoided?
>>    I previously met similar problem on the veth driver, which was latter fixed
>>    by commit  1ce7d306ea63 ("veth: try harder when allocating queue memory"),
>>    where the memory allocating was changed to kvcalloc() to reduces the
>>    possibility of allocation failure. I wonder if similar changes can be applied
>>    to hns3 when allocating memory regions for non-DMA usage.
>>    
> 
> Hi:
> 
> in dmesg, we can see:
> tx_timeout count: 35, queue id: 1, SW_NTU: 0x346, SW_NTC: 0x334, napi state: 17
> BD_NUM: 0x7f HW_HEAD: 0x346, HW_TAIL: 0x346, BD_ERR: 0x0, INT: 0x0
> 
> Because HW_HEAD==HW_TAIL, the hardware has sent all the packets.
> napi state: 17, Therefore, the TX interrupt is received and npai scheduling is triggered.
> However, napi scheduling is not complete, Maybe napi.poll() is not executed.
> Is npai not scheduled in time due to high CPU load in the environment?

Thanks for your analysis. I wonder how can I verify the scheduling of NAPI.

> 
> To solve the memory allocating failure problem,
> you can use kvcalloc to prevent continuous page memory allocating and
> reduce the probability of failure in OOM.

I'm not so familiar with the hns3 driver. I can see several places of memory
allocations and I have no idea which can be replaced and which is required to
be continuous physically. I'll be very happy to test if you can propose a patch.

Cheers,

Miao Wang