linux-kernel - Re: mellanox mlx4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120803083344.GF25979@ics.muni.cz>
Date:	Fri, 3 Aug 2012 10:33:44 +0200
From:	Lukas Hejtmanek <xhejtman@....muni.cz>
To:	Yinghai Lu <yinghai@...nel.org>
Cc:	Roland Dreier <roland@...nel.org>, linux-kernel@...r.kernel.org,
	linux-pci@...r.kernel.org
Subject: Re: mellanox mlx4_core and SR-IOV

On Wed, Aug 01, 2012 at 04:36:14PM -0700, Yinghai Lu wrote:
> > so it seems, that pic=nocsr is a must now.
> 
> yes. Or you have bios provide SRIOV support or 64 bit resource in _CRS.

Well, I can use PCI passthrough in Xen now, however, it seems SR-IOV does not
work in case of Mellanox mlx4 driver.

With 3.5 stock kernel, I got this message in virtual domain:
[    2.666623] mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011)
[    2.666635] mlx4_core: Initializing 0000:00:00.1
[    2.666717] mlx4_core 0000:00:00.1: enabling device (0000 -> 0002)
[    2.666975] mlx4_core 0000:00:00.1: Xen PCI mapped GSI0 to IRQ168
[    2.667040] mlx4_core 0000:00:00.1: enabling bus mastering
[    2.667184] mlx4_core 0000:00:00.1: Detected virtual function - running in slave mode
[    2.667214] mlx4_core 0000:00:00.1: Sending reset
[    2.667319] mlx4_core 0000:00:00.1: Sending vhcr0
[    2.667886] mlx4_core 0000:00:00.1: HCA minimum page size:1
[    2.668067] mlx4_core 0000:00:00.1: The host doesn't support eth interface
[    2.668074] mlx4_core 0000:00:00.1: QUERY_FUNC_CAP command failed, aborting.
[    2.668079] mlx4_core 0000:00:00.1: Failed to obtain slave caps
[    2.668305] mlx4_core: probe of 0000:00:00.1 failed with error -93

not sure what does it mean.

I also tried OFED package from Mellanox which seems to have better SR-IOV
support (at least mlx4_ib does not complain that SR-IOV is not supported).
However, it does not work when SR-IOV enabled:
[13677.034266] mlx4_core 0000:02:00.0: Running in master mode
[13689.278238] mlx4_core 0000:02:00.0: command 0x31 timed out (go bit not cleared)
[13689.278324] mlx4_core 0000:02:00.0: NOP command failed to generate MSI-X interrupt IRQ 241).
[13689.278399] mlx4_core 0000:02:00.0: Trying again without MSI-X.
[13699.286473] mlx4_core 0000:02:00.0: command 0x31 timed out (go bit not cleared)
[13699.286557] mlx4_core 0000:02:00.0: NOP command failed to generate interrupt (IRQ 237), aborting.
[13699.286633] mlx4_core 0000:02:00.0: BIOS or ACPI interrupt routing problem?
[13701.406680] mlx4_core: probe of 0000:02:00.0 failed with error -16

if I disable SR-IOV mode for this driver, it works OK. Could the interrupt
problem be BIOS related? I.e., it won't work until I got BIOS which properly
supports SR-IOV with Mellanox card?

-- 
Lukáš Hejtmánek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/