[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ffc5b2ad-20ba-f6db-610f-f02d788eee0f@intel.com>
Date: Tue, 11 Apr 2023 16:19:13 -0700
From: Jacob Keller <jacob.e.keller@...el.com>
To: Leon Romanovsky <leon@...nel.org>, Jason Gunthorpe <jgg@...dia.com>
CC: Avihai Horon <avihaih@...dia.com>, Aya Levin <ayal@...dia.com>,
"Eric Dumazet" <edumazet@...gle.com>,
Jakub Kicinski <kuba@...nel.org>, <linux-rdma@...r.kernel.org>,
Meir Lichtinger <meirl@...lanox.com>,
"Michael Guralnik" <michaelgur@...lanox.com>,
<netdev@...r.kernel.org>, Paolo Abeni <pabeni@...hat.com>,
Saeed Mahameed <saeedm@...dia.com>,
Shay Drory <shayd@...dia.com>
Subject: Re: [PATCH mlx5-next 4/4] RDMA/mlx5: Allow relaxed ordering read in
VFs and VMs
On 4/10/2023 6:07 AM, Leon Romanovsky wrote:
> From: Avihai Horon <avihaih@...dia.com>
>
> According to PCIe spec, Enable Relaxed Ordering value in the VF's PCI
> config space is wired to 0 and PF relaxed ordering (RO) setting should
> be applied to the VF. In QEMU (and maybe others), when assigning VFs,
> the RO bit in PCI config space is not emulated properly and is always
> set to 0.
>
> Therefore, pcie_relaxed_ordering_enabled() always returns 0 for VFs and
> VMs and thus MKeys can't be created with RO read even if the PF supports
> it.
>
> pcie_relaxed_ordering_enabled() check was added to avoid a syndrome when
> creating a MKey with relaxed ordering (RO) enabled when the driver's
> relaxed_ordering_read_pci_enabled HCA capability is out of sync with FW.
> With the new relaxed_ordering_read capability this can't happen, as it's
> set regardless of RO value in PCI config space and thus can't change
> during runtime.
>
> Hence, to allow RO read in VFs and VMs, use the new HCA capability
> relaxed_ordering_read without checking pcie_relaxed_ordering_enabled().
> The old capability checks are kept for backward compatibility with older
> FWs.
>
> Allowing RO in VFs and VMs is valuable since it can greatly improve
> performance on some setups. For example, testing throughput of a VF on
> an AMD EPYC 7763 and ConnectX-6 Dx setup showed roughly 60% performance
> improvement.
>
> Signed-off-by: Avihai Horon <avihaih@...dia.com>
> Reviewed-by: Shay Drory <shayd@...dia.com>
> Reviewed-by: Aya Levin <ayal@...dia.com>
> Signed-off-by: Leon Romanovsky <leonro@...dia.com>
> ---
Reviewed-by: Jacob Keller <jacob.e.keller@...el.com>
Powered by blists - more mailing lists