lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Wed, 3 Dec 2008 12:20:36 +0100
From:	"Rafael J. Wysocki" <rjw@...k.pl>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	Frans Pop <elendil@...net.nl>, Greg KH <greg@...ah.com>,
	Ingo Molnar <mingo@...e.hu>, jbarnes@...tuousgeek.org,
	lenb@...nel.org,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	tiwai@...e.de, Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: Regression from 2.6.26: Hibernation (possibly suspend) broken on Toshiba R500 (bisected)

On Wednesday, 3 of December 2008, Rafael J. Wysocki wrote:
> On Wednesday, 3 of December 2008, Rafael J. Wysocki wrote:
> > On Wednesday, 3 of December 2008, Linus Torvalds wrote:
> > > 
> > > On Wed, 3 Dec 2008, Rafael J. Wysocki wrote:
> > > > 
> > > > Here's the output of 'lspci -vvxxx':
> > > 
> > > Ok, I'm not finding any documented quirks that would be memory regions, 
> > > and in fact it doesn't look like there is even any remotely likely 32-bit 
> > > valies that might be pointers in your PCI config space that look remotely 
> > > like they might be conflicting in the area of MMIO space that we allocate 
> > > PCI resources from (ie 0x88000000-0x92000000).
> > > 
> > > Of course, any odd MMIO regions might be descibed by some insane model 
> > > that doesn't look like an aligned 32-bit value, but that's unlikely.
> > > 
> > > So I'm still not seeing anything wrong in there.
> > > 
> > > > I'll run the 'pci=cbmemsize=4M' test tomorrow (need to have some sleep).
> > > 
> > > Sure. It will be interesting to see if it makes any difference.
> > 
> > It didn't help (failure in the 4th consecutive hibernation/resume cycle).
> 
> Moreover, to check if the problem may be related to the prefetch window,
> I added the appended patch on top of -rc7 (instead of the previous debug
> patch), which resulted in the following layout in /proc/iomem:
> 
> 88000000-8dffffff : PCI Bus 0000:03
>   88000000-8bffffff : PCI CardBus 0000:04
>   8c000000-8c003fff : 0000:03:0b.1
>   8c004000-8c004fff : 0000:03:0b.0
>     8c004000-8c004fff : yenta_socket
>   8c005000-8c0057ff : 0000:03:0b.1
>     8c005000-8c0057ff : firewire_ohci
>   8c005800-8c0058ff : 0000:03:0b.3
>     8c005800-8c0058ff : mmc0
>   8c400000-8c7fffff : PCI CardBus 0000:04
> 
> and the first hibernation-resume cycle failed.
> 
> Thanks,
> Rafael
> 
> ---
>  drivers/pci/setup-bus.c |   19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> Index: linux-2.6/drivers/pci/setup-bus.c
> ===================================================================
> --- linux-2.6.orig/drivers/pci/setup-bus.c
> +++ linux-2.6/drivers/pci/setup-bus.c
> @@ -353,6 +353,25 @@ static int pbus_size_mem(struct pci_bus 
>  			r_size = resource_size(r);
>  			/* For bridges size != alignment */
>  			align = resource_alignment(r);
> +			if (r->flags & IORESOURCE_PREFETCH) {
> +				resource_size_t expected_align;
> +
> +				expected_align = (i < PCI_BRIDGE_RESOURCES) ?
> +							r_size : r->start;
> +				if (align != expected_align) {
> +					dev_warn(&dev->dev,
> +						"BAR %d [%llx-%llx] "
> +						"alignment issue: flags=%lx "
> +						"align=%llx (%llx)\n", i,
> +						(unsigned long long)r->start,
> +						(unsigned long long)r->end,
> +						r->flags,
> +						(unsigned long long)align,
> +						(unsigned long long)expected_align);
> +					/* Hacky and wrong, but trying to keep things */
> +					align = expected_align;
> +				}
> +			}
>  			order = __ffs(align) - 20;
>  			if (order > 11) {
>  				dev_warn(&dev->dev, "BAR %d bad alignment %llx: "
> --

Now _that_ seems interesting.

I noticed that on this particular box my first patch is equivalent to replacing
IORESOURCE_SIZEALIGN with IORESOURCE_STARTALIGN in two places under the last
'if()' in pci_bus_size_cardbus(), so I wondered what would happen if we only
allocated the prefetchable region in there.  Following that, I did:

--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -432,12 +432,14 @@ static void pci_bus_size_cardbus(struct 
 	 */
 	if (ctrl & PCI_CB_BRIDGE_CTL_PREFETCH_MEM0) {
 		b_res[2].start = 0;
-		b_res[2].end = pci_cardbus_mem_size - 1;
+		b_res[2].end = pci_cardbus_mem_size * 2 - 1;
 		b_res[2].flags |= IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_SIZEALIGN;
 
+#if 0
 		b_res[3].start = 0;
 		b_res[3].end = pci_cardbus_mem_size - 1;
 		b_res[3].flags |= IORESOURCE_MEM | IORESOURCE_SIZEALIGN;
+#endif
 	} else {
 		b_res[3].start = 0;
 		b_res[3].end = pci_cardbus_mem_size * 2 - 1;

(well, I didn't have to allocate twice the size but that was to check if the
size of the allocation was going to matter), which resulted in the appended
/proc/iomem.

Then, voila!, I'm not able to reproduce the hibernation-resume failure.

This appears to mean that:
(1) The sizes of the allocations and the locations of devices in the memory
    address space don't matter here.
(2) The presence and size of the prefetchable memory window don't matter here.
(3) What matters is the presence of non-prefetchable memory window on the
    supposedly transparent bridge.  Namely, if the window is there, resume from
    hibernation occasionally fails (again, the size of the window and the
    location of it in the memory address space doesn't seem to matter).

So, apparently, on this box (and I guess on Frans' too) we could avoid the
problem if we didn't allocate the non-prefetchable memory window in
pci_bus_size_cardbus(), but I guess that wouldn't be generally correct.

Also, I would be happy to actually understand _why_ this happens.

Thanks,
Rafael

---
00000000-0009fbff : System RAM
0009fc00-0009ffff : reserved
000e0000-000eedff : reserved
000eee00-000eefff : ACPI Non-volatile Storage
000ef000-000fffff : reserved
00100000-7f74ffff : System RAM
  00200000-00450486 : Kernel code
  00450487-00603c37 : Kernel data
  00678000-00784d3b : Kernel bss
7f750000-7fffffff : reserved
  7f750000-7f75ffff : pnp 00:00
  7f760000-7f7fffff : pnp 00:00
  7f800000-7fffffff : pnp 00:00
88000000-8fffffff : PCI Bus 0000:03 <-- the prefetchable window
  88000000-8fffffff : PCI CardBus 0000:04 <-- the prefetchable window
90000000-900fffff : PCI Bus 0000:03
  90000000-90003fff : 0000:03:0b.1
  90004000-90004fff : 0000:03:0b.0
    90004000-90004fff : yenta_socket
  90005000-900057ff : 0000:03:0b.1
    90005000-900057ff : firewire_ohci
  90005800-900058ff : 0000:03:0b.3
    90005800-900058ff : mmc0
90100000-9017ffff : 0000:00:02.1
90180000-90183fff : 0000:00:1b.0
  90180000-90183fff : ICH HD audio
90184000-90184fff : Intel Flush Page
90400000-907fffff : PCI CardBus 0000:04 <-- last interesting line
e0000000-efffffff : 0000:00:02.0
  e0000000-e07affff : vesafb
f0000000-f3ffffff : PCI MMCONFIG 0
  f0000000-f3ffffff : pnp 00:01
fec00000-fec17fff : reserved
  fec00000-fec17fff : pnp 00:00
    fec00000-fec00fff : IOAPIC 0
fec20000-fec27fff : reserved
  fec20000-fec27fff : pnp 00:00
fed00000-fed003ff : HPET 0
  fed00000-fed003ff : reserved
fed14000-fed19fff : reserved
  fed14000-fed19fff : pnp 00:00
fed1c000-fed8ffff : reserved
  fed1c000-fed1ffff : pnp 00:00
  fed20000-fed3ffff : pnp 00:00
  fed45000-fed8ffff : pnp 00:00
feda0000-fedbffff : reserved
  feda0000-fedbffff : pnp 00:00
fee00000-fee00fff : Local APIC
  fee00000-fee00fff : reserved
    fee00000-fee00fff : pnp 00:00
ff800000-ff8fffff : PCI Bus 0000:02
  ff8fe000-ff8fffff : 0000:02:00.0
    ff8fe000-ff8fffff : iwlagn
ff900000-ff9fffff : PCI Bus 0000:01
  ff9e0000-ff9fffff : 0000:01:00.0
    ff9e0000-ff9fffff : e1000e
ffa00000-ffbfffff : reserved
  ffa00000-ffbfffff : pnp 00:00
ffc3f800-ffc3fbff : 0000:00:1f.2
  ffc3f800-ffc3fbff : ahci
ffc3fc00-ffc3ffff : 0000:00:1d.7
  ffc3fc00-ffc3ffff : ehci_hcd
ffc40000-ffc7ffff : 0000:00:02.0
ffc80000-ffcfffff : 0000:00:02.0
ffd00000-ffffffff : reserved
  ffd00000-ffffffff : pnp 00:00
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ