netdev - Re: [PATCH v3 1/1] net/mlx5: Added cond

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <0d7deb6e-f12d-4cd9-97f5-2661ddaab25a@nvidia.com>
Date: Thu, 19 Sep 2024 20:48:20 +0300
From: Moshe Shemesh <moshe@...dia.com>
To: Mohamed Khalfella <mkhalfella@...estorage.com>, Tariq Toukan
	<tariqt@...dia.com>, Alexander Lobakin <aleksander.lobakin@...el.com>, "Saeed
 Mahameed" <saeedm@...dia.com>, Leon Romanovsky <leon@...nel.org>, "David S.
 Miller" <davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>, "Jakub
 Kicinski" <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>
CC: <yzhong@...estorage.com>, Przemek Kitszel <przemyslaw.kitszel@...el.com>,
	Shay Drori <shayd@...dia.com>, <netdev@...r.kernel.org>,
	<linux-rdma@...r.kernel.org>, <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v3 1/1] net/mlx5: Added cond_resched() to crdump
 collection



On 9/5/2024 7:02 AM, Mohamed Khalfella wrote:
> Collecting crdump involves reading vsc registers from pci config space
> of mlx device, which can take long time to complete. This might result
> in starving other threads waiting to run on the cpu.
> 
> Numbers I got from testing ConnectX-5 Ex MCX516A-CDAT in the lab:
> 
> - mlx5_vsc_gw_read_block_fast() was called with length = 1310716.
> - mlx5_vsc_gw_read_fast() reads 4 bytes at a time. It was not used to
>    read the entire 1310716 bytes. It was called 53813 times because
>    there are jumps in read_addr.
> - On average mlx5_vsc_gw_read_fast() took 35284.4ns.
> - In total mlx5_vsc_wait_on_flag() called vsc_read() 54707 times.
>    The average time for each call was 17548.3ns. In some instances
>    vsc_read() was called more than one time when the flag was not set.
>    As expected the thread released the cpu after 16 iterations in
>    mlx5_vsc_wait_on_flag().
> - Total time to read crdump was 35284.4ns * 53813 ~= 1.898s.
> 
> It was seen in the field that crdump can take more than 5 seconds to
> complete. During that time mlx5_vsc_wait_on_flag() did not release the
> cpu because it did not complete 16 iterations. It is believed that pci
> config reads were slow. Adding cond_resched() every 128 register read
> improves the situation. In the common case the, crdump takes ~1.8989s,
> the thread yields the cpu every ~4.51ms. If crdump takes ~5s, the thread
> yields the cpu every ~18.0ms.
> 
> Fixes: 8b9d8baae1de ("net/mlx5: Add Crdump support")
> Reviewed-by: Yuanyuan Zhong<yzhong@...estorage.com>
> Signed-off-by: Mohamed Khalfella<mkhalfella@...estorage.com>
> ---

Reviewed-by: Moshe Shemesh <moshe@...dia.com>