netdev - Re: [PATCH] PCI: Add no-D3 quirk for Mellanox ConnectX-[45]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <82b867e0-25b5-622f-2735-cfe9df6cdb37@ozlabs.ru>
Date:   Wed, 9 Jan 2019 19:14:25 +1100
From:   Alexey Kardashevskiy <aik@...abs.ru>
To:     Benjamin Herrenschmidt <benh@...nel.crashing.org>,
        David Gibson <david@...son.dropbear.id.au>
Cc:     Jason Gunthorpe <jgg@...pe.ca>, Leon Romanovsky <leon@...nel.org>,
        davem@...emloft.net, saeedm@...lanox.com, ogerlitz@...lanox.com,
        tariqt@...lanox.com, bhelgaas@...gle.com,
        linux-kernel@...r.kernel.org, linuxppc-dev@...ts.ozlabs.org,
        netdev@...r.kernel.org, alex.williamson@...hat.com,
        linux-pci@...r.kernel.org, linux-rdma@...r.kernel.org,
        sbest@...hat.com, paulus@...ba.org
Subject: Re: [PATCH] PCI: Add no-D3 quirk for Mellanox ConnectX-[45]



On 09/01/2019 18:25, Benjamin Herrenschmidt wrote:
> On Wed, 2019-01-09 at 17:32 +1100, Alexey Kardashevskiy wrote:
>> I have just moved the "Mellanox Technologies MT27700 Family
>> [ConnectX-4]" from garrison to firestone machine and there it does not
>> produce an EEH, with the same kernel and skiboot (both upstream + my
>> debug). Hm. I cannot really blame the card but I cannot see what could
>> cause the difference in skiboot either. I even tried disabling NPU so
>> garrison would look like firestone, still EEH'ing.
> 
> The systems have a different chip though, firestone is P8 and garrison
> is P8', which a slightly different PHB revision. Worth checking if we
> have anything significantly different in our inits and poke at the HW
> guys.

Nope, we do not have anything different for these machines. Asking HW
guys never worked for me :-/

I think the easiest is just doing what we did for PHB4 and ignoring
these D3 requests on garrisons.


> BTW. Are the cards behind a switch in either case ?


No, directly connected to the root on both:

garrison:

0000:00:00.0 PCI bridge: IBM Device 03dc (rev ff)
0000:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4] (rev ff)
0000:01:00.1 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4] (rev ff)

firestone (phb #0 is taken by nvidia gpu):

0001:00:00.0 PCI bridge: IBM POWER8 Host Bridge (PHB3)
0001:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]
0001:01:00.1 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]


-- 
Alexey