lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <2673153f-3330-4a02-8bf0-ee1727715381@oracle.com>
Date: Mon, 1 Jul 2024 11:10:58 +0530
From: Anand Khoje <anand.a.khoje@...cle.com>
To: David Laight <David.Laight@...LAB.COM>,
        Jesse Brandeburg <jesse.brandeburg@...el.com>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Cc: "saeedm@...lanox.com" <saeedm@...lanox.com>,
        "leon@...nel.org" <leon@...nel.org>,
        "tariqt@...dia.com"
 <tariqt@...dia.com>,
        "edumazet@...gle.com" <edumazet@...gle.com>,
        "kuba@...nel.org" <kuba@...nel.org>,
        "pabeni@...hat.com"
 <pabeni@...hat.com>,
        "davem@...emloft.net" <davem@...emloft.net>
Subject: Re: [PATCH v5] net/mlx5: Reclaim max 50K pages at once


On 6/28/24 21:14, David Laight wrote:
> ...
>> The way Mellanox ConnectX5 driver handles 'release of allocated pages
>> from HCA' or 'allocation of pages to HCA', is by sending an event to the
>> host. This event will have number of pages in it. If the number is
>> positive, that indicates HCA is requesting that number of pages to be
>> allocated. And if that number is negative, it is the HCA indicating that
>> that number of pages can be reclaimed by the host.
> A one line comment would do.
> Possibly even negating the be32toh() result?
>
>> In this patch we are restricting the maximum number of pages that can be
>> reclaimed to be 50000 (effectively this would be -50000 as it is
>> reclaim). This limit is based on the capability of the firmware as it
>> cannot release more than 50000 back to the host in one go.
> Hang on, why are you soft limiting it to the hard limit?
> I thought the problem was that releasing a lot of pages took a long
> time and 'stuffed' other time-critical tasks.
>
> The only way to resolve that would seem to be to defer the actual freeing
> to a low (or at least normal user) priority thread.
> You would definitely want to get out of 'softint' context.
> (Which is out of napi unless forced to be threaded - and that only really
> works if you force the threads under the RT scheduler.)
>
> 	David

Hi David,

The issue here is, when Mellanox device sends a huge number of pages 
back to the host to reclaim, the host allocates a certain number of 
mailbox messages mlx5_cmd_mailbox to accommodate the DMA addresses of 
the memory to be reclaimed. The freeing of these mailbox messages is 
time consuming (not the freeing of actual pages).

Now, the limit of the FW is that presently, it frees upto 50000 pages. 
This limit can increase in future firmware versions. We are limiting 
this in the driver because we see optimal results with this limit during 
our tests. The results indicated that the time consumed while freeing of 
mailbox messages stayed 2 usec on average - which is tolerable and would 
not need running this thread in a different (low priority) context.

Thanks,

Anand

> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ