linux-kernel - x86-32 PAE-enabled kernel fails to boot with 64GB of RAM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 6 Dec 2011 02:23:26 -0500
From:	Arnaud Lacombe <lacombar@...il.com>
To:	x86@...nel.org, LKML <linux-kernel@...r.kernel.org>
Subject: x86-32 PAE-enabled kernel fails to boot with 64GB of RAM

Hi,

I've been playing a bit with PAE and 3.2-rc3 this evening and found
out to be unable to use the full 64GB address space. More precisely,
the attached config, fails to detect the disk correctly:

excerpt from `dmesg.64GB.txt' (bad):

scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    0.14 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 107856 512-byte logical blocks: (55.2 MB/52.6 MiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
support DPO or FUA
 sda: unknown partition table
sd 0:0:0:0: Attached scsi generic sg0 type 0

excerpt from `dmesg.16GB.txt' (good):

scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    0.14 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 107856 512-byte logical blocks: (55.2 MB/52.6 MiB)
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
support DPO or FUA
 sda: sda1 sda2

The theoretical use-case I would think about is a machine running an
x86-64 kernel with +64GB of RAM, but used to boot an x86-32 kernel
every once in a while to do some work. I would naively expect such a
PAE-enabled kernel to be limited to 64GB of RAM, but still be
functional.

The command used to trigger the issue is:

# qemu -nographic \
    -mem-prealloc -mem-path $PWD/ramdisk -m 128G \
    -kernel arch/x86/boot/bzImage \
    -append 'root=/dev/sda2 rootfstype=ext4 rootwait console=ttyS0
init=/etc/preinit loglevel=8' \
    -hda openwrt-x86-generic-combined-ext4.img

any hints ?

Thanks,
 - Arnaud

ps: instability starts to appear consistently above 65000MB of RAM, at
65025MB, the kernel BUG() on the following:

PCI: Using ACPI for IRQ routing^M
PCI: pci_cache_line_size set to 32 bytes^M
reserve RAM buffer: 000000000009f400 - 000000000009ffff ^M
reserve RAM buffer: 00000000dfffd000 - 00000000dfffffff ^M
reserve RAM buffer: 0000001000100000 - 0000001003ffffff ^M
BUG: unable to handle kernel paging request at 1f90c2f2^M
IP: [<c1003cbf>] print_context_stack+0x6e/0x8d^M
*pdpt = 0000000000000000 *pde = 0000000000000000 ^M
Thread overran stack, or stack corrupted^M
Oops: 0000 [#1] DEBUG_PAGEALLOC^M
Modules linked in:^M
^M
Pid: 1, comm: swapper Not tainted 3.2.0-rc3 #5 Bochs Bochs^M
EIP: 0060:[<c1003cbf>] EFLAGS: 00000097 CPU: 0^M
EIP is at print_context_stack+0x6e/0x8d^M
EAX: ffffe000 EBX: 1f90c2f2 ECX: 00000000 EDX: 1f90c2f2^M
ESI: 00000000 EDI: 00000000 EBP: d748c118 ESP: d748c0f8^M
 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068^M
Process swapper (pid: 1, ti=d748a000 task=d7490000 task.ti=d748c000)^M
Stack:^M
 00000000 ffffe000 1f90dffc c100981f 1f90c000 1f90c2f2 00000000 c1212998^M
 d748c148 c100321a c1212998 d748c168 00000000 d748c134 1f90c000 00000000^M
 c106a857 d748c168 c14864ad 00000044 d748c160 c10097ae 00000000 c1212998^M
Call Trace:^M
 [<c100981f>] ? save_stack_address_nosched+0x15/0x15^M
 [<c100321a>] dump_trace+0x78/0xa5^M
 [<c106a857>] ? check_bytes_and_report+0x21/0xa7^M
 [<c14864ad>] ? __reserve_region_with_split+0x5d/0xd0^M
 [<c10097ae>] save_stack_trace+0x1c/0x38^M
 [<c106a00b>] set_track+0x48/0xa3^M
 [<c1208298>] free_debug_processing+0xe9/0x15a^M
 [<c14864ad>] ? __reserve_region_with_split+0x5d/0xd0^M
 [<c12083dc>] __slab_free+0x35/0x1c4^M
 [<c14864ad>] ? __reserve_region_with_split+0x5d/0xd0^M
 [<c103cac3>] ? mark_held_locks+0x59/0x77^M
 [<c106a0ff>] ? slab_free_hook+0x33/0x3a^M
 [<c103cbf4>] ? trace_hardirqs_on_caller.part.19+0xab/0xdb^M
 [<c106a0ff>] ? slab_free_hook+0x33/0x3a^M
 [<c103cc87>] ? trace_hardirqs_on_caller+0x63/0x66^M
 [<c106b81b>] kfree+0x9b/0xa4^M
 [<c106b81b>] ? kfree+0x9b/0xa4^M
 [<c14864ad>] ? __reserve_region_with_split+0x5d/0xd0^M
 [<c14864ad>] ? __reserve_region_with_split+0x5d/0xd0^M
 [<c14864ad>] __reserve_region_with_split+0x5d/0xd0^M
 [<c1486515>] __reserve_region_with_split+0xc5/0xd0^M

but I would assume that such memory value (odd, non-multiple of 4 or
8) is not supported.

View attachment "config.txt" of type "text/plain" (37326 bytes)

View attachment "dmesg.16GB.txt" of type "text/plain" (13150 bytes)

View attachment "dmesg.64GB.txt" of type "text/plain" (12964 bytes)