Message-ID: <alpine.LFD.2.00.0812041459120.3543@nehalem.linux-foundation.org>
Date:	Thu, 4 Dec 2008 15:22:30 -0800 (PST)
From:	Linus Torvalds <torvalds@...ux-foundation.org>
To:	"Rafael J. Wysocki" <rjw@...k.pl>
cc:	Frans Pop <elendil@...net.nl>, Greg KH <greg@...ah.com>,
	Ingo Molnar <mingo@...e.hu>, jbarnes@...tuousgeek.org,
	lenb@...nel.org,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	tiwai@...e.de, Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: Regression from 2.6.26: Hibernation (possibly suspend) broken
 on Toshiba R500 (bisected)



On Thu, 4 Dec 2008, Rafael J. Wysocki wrote:
> 
> That said the "don't bother allocating bridging windows for transparent
> bridges" patch resulted in the following layout on my box (from /proc/iomem):
> 
> 88000000-8807ffff : 0000:00:02.1
> 88080000-88083fff : 0000:00:1b.0
>   88080000-88083fff : ICH HD audio
> 88084000-88087fff : 0000:03:0b.1
> 88088000-88088fff : 0000:03:0b.0
>   88088000-88088fff : yenta_socket
> 88089000-880897ff : 0000:03:0b.1
>   88089000-880897ff : firewire_ohci
> 88089800-880898ff : 0000:03:0b.3
>   88089800-880898ff : mmc0
> 8808a000-8808afff : Intel Flush Page
> 8c000000-8fffffff : PCI CardBus 0000:04
> 90000000-93ffffff : PCI CardBus 0000:04
> 
> while my "don't allocate bridging windows for cardbus bridges behind
> transparent bridges" patch I've just sent (appended for easier reference)
> results in the layout:
> 
> 88000000-880fffff : PCI Bus 0000:03
>   88000000-88003fff : 0000:03:0b.1
>   88004000-88004fff : 0000:03:0b.0
>     88004000-88004fff : yenta_socket
>   88005000-880057ff : 0000:03:0b.1
>     88005000-880057ff : firewire_ohci
>   88005800-880058ff : 0000:03:0b.3
>     88005800-880058ff : mmc0
> 88100000-8817ffff : 0000:00:02.1
> 88180000-88183fff : 0000:00:1b.0
>   88180000-88183fff : ICH HD audio
> 88184000-88184fff : Intel Flush Page
> 8c000000-8fffffff : PCI CardBus 0000:04
> 90000000-93ffffff : PCI CardBus 0000:04

Well, this happens because in the second case, you still will allocate a 
non-prefetchable window due to the _other_ devices behind the bridge (ie 
the firewire and mmc device.

So with your "ignore cardbus device resources", you still do end up with a 
bridge window allocated, but now it will depend entirely on what other 
devices exist on that PCI bus. Which is why I really dislike that patch, 
because it really makes no sense at all. 

And once you've allocated the PCI bridge window, then the alignment 
requirements are that you end up having at least 1MB of window, and now 
because a window exists, and will fit, it will then happily put the yenta 
control mappings into that window even though it didn't size it for them.

See?

Also, notice how it doesn't put the actual cardbus bridge windows 
themselves into that PCI bridge window, because they won't fit. So these 
resources:

	8c000000-8fffffff : PCI CardBus 0000:04
	90000000-93ffffff : PCI CardBus 0000:04

are outside the window, even though that bus is topologically inside the 
same bus, and they both end up depending on the fact that the PCI bus 
controller is transparent.

So the above results are fairly easy to explain.

The _ordering_ difference comes simply from the allocation order. We'll do 
bus allocations first (if we do them), which is why _if_ we allocate a 
window for that PCI bus 03 (ie the one that is bridged to by 0:1e.0, that 
transparent bridge), then we'll end up allocating it first. So that's why 
the PCI bridge window shows up at 0x88000000, if it shows up at all 
(because that's the starting address for PCI allocations).

Then, we'll do the regular devices in the order we found them, so then 
we'll allocate the resources for device 0:02.1 and then 0:1b.0. Now, _if_ 
we did a bus window allocation, they'll end up being after the bus window 
we allocated. Otherwise they'll end up being the first allocations.

So then, by the time we actually get to bus#3, and devices 3:0b.*, where 
they end up is going to depend on whether we allocated that bus window (in 
which case they'll preferentially get allocated inside the window - ie 
starting at 0x88000000), or they'll just get allocated inside the parent 
(root) PCI bus. In the latter case they'll be allocated after the 0:02.1 
etc devices.

So the differences in ordering really do make sense, and are a direct 
consequence of whether we decided to need to allocate a bridging window 
for that PCI-PCI bridge at 0:1e.0.

Also note that _if_ PCI bus #3 had had other devices with prefetchable 
memory resources, then we'd have allocated a prefetchable window for 
those even with your patch, and then we'd have been back to the original 
allocation again (although sizing might cause changes, since your patch 
will make us ignore the cardbus controlle for sizing).

> where devices behind the transparent bridge (PCI Bus 0000:03) are located
> _before_ ICH HD audio in the memory address space, and this one appears to
> work.  So there _may_ be an effect of the layout too.

I do agree that your patch will affect layout. I just don't think it makes 
any real amount of sense because of how it essentially does so "randomly" 
depending on what other devices you'd have behind the transparent bridge.

Which is why I think we should either not size transparent bridges at all, 
or we should size _all_ the devices behind them.

Oh, btw, I thought your and Frans' laptops were identical? They don't seem 
to be: in Frans' case, the firmware does seem to set up one memory window, 
so he gets that one even with my "don't size anything" patch, just because 
we'll still honor allocations done by firmware.

> Well, given that both affected boxes have the same chipset (945GM), I seriously
> suspect a nastiness in that chipset we're not aware of.  Especially that
> the problem is not reproducible without snd_hda_intel (at least on my box).

Well, the counter-argument to that is that the 945GM should be a _very_ 
common chipset. So I'd not expect a chipset nastiness.

I'd be more wary about the BIOS doing something odd. Or an ACPI thing. 

> I've already checked the resume ordering of PCI devices on my box and it
> is the following:
>   [snip]
> So, snd_hda_intel resumes before all of the bridges and the layout of devices
> _behind_ the transparent bridge shouldn't affect it at all.

Yes. And in the case of Frans' machine, the e1000e controller was before 
all the bridges too.

		Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists