Date:	Thu, 11 Sep 2014 12:26:54 -0700
From:	Yinghai Lu <yinghai@...nel.org>
To:	Bjorn Helgaas <bhelgaas@...gle.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	Dirk Gouders <dirk@...ders.net>,
	Andreas Noever <andreas.noever@...il.com>,
	Linux Kernel <linux-kernel@...r.kernel.org>,
	"linux-pci@...r.kernel.org" <linux-pci@...r.kernel.org>
Subject: Re: [BUG] Bisected Problem with LSI PCI FC Adapter

On Thu, Sep 11, 2014 at 10:30 AM, Bjorn Helgaas <bhelgaas@...gle.com> wrote:
> [+cc linux-pci]
>
>
> On Thu, Sep 11, 2014 at 7:43 AM, Dirk Gouders <dirk@...ders.net> wrote:
>> Andreas Noever <andreas.noever@...il.com> writes:
>>
>>> On Wed, Sep 3, 2014 at 2:47 PM, Dirk Gouders <dirk@...ders.net> wrote:
>>>> Andreas Noever <andreas.noever@...il.com> writes:
>>>>
>>>>> On Wed, Sep 3, 2014 at 12:57 PM, Dirk Gouders <dirk@...ders.net> wrote:
>>>>>> On a Tyan VX50 (B4985) I ran into problems when updating the kernel: the
>>>>>> PCI FC Adapter is no longer recognized.
>>>>>
>>>>> Can you provide the output of lspci -vvv and the output of dmesg from
>>>>> a working boot? Which card is the one that is not recognized?
>>>>
>>>> Sure, the card that disappeared is:
>>>>
>>>> 0a:00.0 Fibre Channel: LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter (rev 02)
>>>
>>> As far as I can tell the following is happening:
>>> The root bus resource window (advertised by the bios?) is to small:
>>> pci_bus 0000:00: root bus resource [bus 00-07]
>>> Previously we didn't really care. There is a resource conflict but we
>>> ignored it:
>>> pci_bus 0000:0a: busn_res: can not insert [bus 0a] under [bus 00-07]
>>> (conflicts with (null) [bus 00-07])
>>> With the patch we mark the bridge as broken and reassign the bus to 06:
>>> pci 0000:00:0e.0: bridge configuration invalid ([bus 0a-0a]), reconfiguring
>>> pci 0000:00:0e.0: PCI bridge to [bus 06-07]
>>> pci 0000:00:0e.0:   bridge window [io  0x3000-0x3fff]
>>> pci 0000:00:0e.0:   bridge window [mem 0xd4200000-0xd42fffff]
>>> pci_bus 0000:06: busn_res: [bus 06-07] end is updated to 06

> Thanks for following up on this.  It had fallen off my radar, so I
> opened https://bugzilla.kernel.org/show_bug.cgi?id=84281 to make sure
> I don't forget again.  Please continue the debug discussion here in
> email.

Two problems here:
1. This is amd two node systems. amd_bus.c tell us bus [00, 7f] is from
first socket, but _OSC says only [0,7] is from first socket.

So solution (1):
According to Linus's principle, we should always trust HW than firmware,
so should we just adjust bus range from _OSC before we use it?

2. After moving, LSI FC card from bus 0a to bus 07, the LSI refuse to respond.

During my testing with pci busn allocation patchset, I found that if changing
LSI Erie card to different bus, it will refuse to responding. Only
thing that will
make the LSI card again, is resetting the pcie link. This should be LSI firmware
bug.

Dirk, please check if you can apply attached patches to use

echo 1 > /sys/bus/pci/devices/0000\:00\0e.0/link_disable
echo 0 > /sys/bus/pci/devices/0000\:00\0e.0/link_disable

to reset the link.

Solution (2)
To workaround the problem, we could reset the pcie link after change bus num
in the pcie bridges ?

Soultion (3)
Or we just revert the offending 1820ffdccb9b4398 (PCI: Make sure
bus number resources stay within their parents bounds) ?

Thanks

Yinghai

View attachment "move_pcie_link_disable_1.patch" of type "text/x-patch" (2628 bytes)

View attachment "move_pcie_link_disable_2.patch" of type "text/x-patch" (1947 bytes)

View attachment "pci_express_link.patch" of type "text/x-patch" (2574 bytes)

View attachment "pci_express_link_disable.patch" of type "text/x-patch" (1930 bytes)

Powered by blists - more mailing lists