netdev - Re: [PATCH net-next v2 1/2] mlx5/core: relax memory barrier in eq_update

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f1a56268-f1d5-4204-be34-5c7a1749bcd9@gmail.com>
Date: Mon, 11 Nov 2024 13:59:57 +0200
From: Tariq Toukan <ttoukan.linux@...il.com>
To: Parav Pandit <parav@...dia.com>,
 Caleb Sander Mateos <csander@...estorage.com>,
 Saeed Mahameed <saeedm@...dia.com>, Leon Romanovsky <leon@...nel.org>,
 Tariq Toukan <tariqt@...dia.com>, Andrew Lunn <andrew+netdev@...n.ch>,
 "David S. Miller" <davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>,
 Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>
Cc: "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
 "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
 "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH net-next v2 1/2] mlx5/core: relax memory barrier in
 eq_update_ci()



On 08/11/2024 12:46, Parav Pandit wrote:
> 
>> From: Caleb Sander Mateos <csander@...estorage.com>
>> Sent: Friday, November 8, 2024 12:01 AM
>>
>> The memory barrier in eq_update_ci() after the doorbell write is a significant
>> hot spot in mlx5_eq_comp_int(). Under heavy TCP load, we see 3% of CPU
>> time spent on the mfence instruction.
>>
>> 98df6d5b877c ("net/mlx5: A write memory barrier is sufficient in EQ ci
>> update") already relaxed the full memory barrier to just a write barrier in
>> mlx5_eq_update_ci(), which duplicates eq_update_ci(). So replace mb() with
>> wmb() in eq_update_ci() too.
>>
>> On strongly ordered architectures, no barrier is actually needed because the
>> MMIO writes to the doorbell register are guaranteed to appear to the device in
>> the order they were made. However, the kernel's ordered MMIO primitive
>> writel() lacks a convenient big-endian interface.
>> Therefore, we opt to stick with __raw_writel() + a barrier.
>>
>> Signed-off-by: Caleb Sander Mateos <csander@...estorage.com>
>> ---
>> v2: keep memory barrier instead of using ordered writel()
>>
>>   drivers/net/ethernet/mellanox/mlx5/core/lib/eq.h | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/eq.h
>> b/drivers/net/ethernet/mellanox/mlx5/core/lib/eq.h
>> index 4b7f7131c560..b1edc71ffc6d 100644
>> --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/eq.h
>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/eq.h
>> @@ -70,11 +70,11 @@ static inline void eq_update_ci(struct mlx5_eq *eq, int
>> arm)
>>   	__be32 __iomem *addr = eq->doorbell + (arm ? 0 : 2);
>>   	u32 val = (eq->cons_index & 0xffffff) | (eq->eqn << 24);
>>
>>   	__raw_writel((__force u32)cpu_to_be32(val), addr);
>>   	/* We still want ordering, just not swabbing, so add a barrier */
>> -	mb();
>> +	wmb();
>>   }
>>
>>   int mlx5_eq_table_init(struct mlx5_core_dev *dev);  void
>> mlx5_eq_table_cleanup(struct mlx5_core_dev *dev);  int
>> mlx5_eq_table_create(struct mlx5_core_dev *dev);
>> --
>> 2.45.2
> 
> Reviewed-by: Parav Pandit <parav@...dia.com>
> 

Acked-by: Tariq Toukan <tariqt@...dia.com>