Message-ID: <20230926010159.0f25161c@yea>
Date:   Tue, 26 Sep 2023 01:01:59 +0200
From:   Erhard Furtner <erhard_f@...lbox.org>
To:     linuxppc-dev@...ts.ozlabs.org
Cc:     linux-kernel@...r.kernel.org, Liam.Howlett@...cle.com
Subject: [Bisected] PowerMac G4 getting "BUG: Unable to handle kernel data
 access on write at 0x00001ff0" at boot with CONFIG_VMAP_STACK=y on kernels
 6.5.x (regression over 6.4.x)

Greetings!

Had a chat on #gentoo-powerpc with another user whose G4 Mini fails booting kernel 6.5.0 when CONFIG_VMAP_STACK=y is enabled. I was able to replicate the issue on my PowerMac G4. Also I was able to bisect the issue.

Kernels 6.4.x boot ok with CONFIG_VMAP_STACK=y but on 6.5.5 I get:

[...]
Kernel attempted to write user page (1ff0) - exploit attempt? (uid: 0)
BUG: Unable to handle kernel data access on write at 0x00001ff0
Faulting instruction address: 0xc0008750
Oops: Kernel access of bad area, sig: 11 [#1]
BE PAGE_SIZE=4K MMU=Hash PowerMac
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 6.5.5-PMacG4 #5
Hardware name: PowerMac3,6 7455 0x80010303 PowerMac
NIP:  c0008750 LR: c0041848 CTR: c0070988
REGS: c0d3dcd0 TRAP: 0300   Not tainted (6.5.5-PMacG4)
MSR:  00001032 <ME,IR,DR,RI>  CR: 22d3ddc0 XER: 20000000
DAR: 00001ff0 DSISR: 42000000
GPR00: c0041848 c0d3dd90 c0d06360 c0d3ddd0 c0d06360 c0d3dea8 c0d3adc0 00000000
GPR08: 00000000 c0d40000 00000000 c0d3ddc0 00000000 00000000 00000000 00000004
GPR16: 00000002 00000000 00000002 00402dc2 00402dc2 00002000 f1004000 00000000
GPR24: c0d45220 c0d06644 c0843c34 00000002 c0d06360 c0d0ce00 c0d06360 00000000
NIP [c0008750] do_softirq_own_stack+0x18/0x3c
LR [c0041848] irq_exit+0x98/0xc4
Call Trace:
[c0d3dd90] [c0d69564] 0xc0d69564 (unreliable)
[c0d3ddb0] [c0041848] irq_exit+0x98/0xc4
[c0d3ddc0] [c0004a98] Decrementer_virt+0x108/0x10c
--- interrupt: 900 at __schedule+0x43c/0x4e0
NIP:  c0843940 LR: c084398c CTR: c0070988
REGS: c0d3ddd0 TRAP: 0900   Not tainted  (6.5.5-PMacG4)
MSR:  00009032 <EE,ME,IR,DR,RI>  CR: 22024484  XER: 00000000

GPR00: c0843574 c0d3de90 c0d06360 c0d06360 c0d06360 c0d3dea8 00000001 00000000
GPR08: 00000000 00009032 c099ce2c 0007ffbf 22024484 00000000 00000000 00000004
GPR16: 00000002 00000000 00000002 00402dc2 00402dc2 00002000 f1004000 00000000
GPR24: c0d45220 c0d06644 c0843c34 00000002 c0d06360 c0d0ce00 c0d06360 c0d063ac
NIP [c0843940] __schedule+0x43c/0x4e0
LR [c084390c] __schedule+0x408/0x4e0
--- interrupt: 900
[c0d3de90] [c0843574] __schedule+0x70/0x4e0 (unreliable)
[c0d3ded0] [c0843c34] __cond_resched+0x34/0x54
[c0d3dee0] [c0141068] __vmalloc_node_range+0x27c/0x64c
[c0d3de60] [c0141794] __vmalloc_node+0x44/0x54
[c8d3df80] [c0c06510] init_IRQ+0x34/0xd4
[c8d3dfa0] [c0c03440] start_kernel+0x424/0x558
[c8d3dff0] [00003540] 0x3540
Code: 39490999 7d4901a4 39290aaa 7d2a01a4 4c00012c 4bffff20 9421ffe0 7c08002a6 3d20c0d4 93e1001c 90010024 83e95278 <943f1ff0> 7fe1fb78 48840c6d 80210000
---[ end trace 0000000000000000 ]---

Kernel panic - not syncing: Attempted to kill the idle task!
Rebooting in 48 seconds..


The bisect revealed this commit:
 # git bisect good
cfeb6ae8bcb96ccf674724f223661bbcef7b0d0b is the first bad commit
commit cfeb6ae8bcb96ccf674724f223661bbcef7b0d0b
Author: Liam R. Howlett <Liam.Howlett@...cle.com>
Date:   Fri Aug 18 20:43:55 2023 -0400

    maple_tree: disable mas_wr_append() when other readers are possible
    
    The current implementation of append may cause duplicate data and/or
    incorrect ranges to be returned to a reader during an update.  Although
    this has not been reported or seen, disable the append write operation
    while the tree is in rcu mode out of an abundance of caution.
    
    During the analysis of the mas_next_slot() the following was
    artificially created by separating the writer and reader code:
    
    Writer:                                 reader:
    mas_wr_append
        set end pivot
        updates end metata
        Detects write to last slot
        last slot write is to start of slot
        store current contents in slot
        overwrite old end pivot
                                            mas_next_slot():
                                                    read end metadata
                                                    read old end pivot
                                                    return with incorrect range
        store new value
    
    Alternatively:
    
    Writer:                                 reader:
    mas_wr_append
        set end pivot
        updates end metata
        Detects write to last slot
        last lost write to end of slot
        store value
                                            mas_next_slot():
                                                    read end metadata
                                                    read old end pivot
                                                    read new end pivot
                                                    return with incorrect range
        set old end pivot
    
    There may be other accesses that are not safe since we are now updating
    both metadata and pointers, so disabling append if there could be rcu
    readers is the safest action.
    
    Link: https://lkml.kernel.org/r/20230819004356.1454718-2-Liam.Howlett@oracle.com
    Fixes: 54a611b60590 ("Maple Tree: add new data structure")
    Signed-off-by: Liam R. Howlett <Liam.Howlett@...cle.com>
    Cc: <stable@...r.kernel.org>
    Signed-off-by: Andrew Morton <akpm@...ux-foundation.org>

 lib/maple_tree.c | 7 +++++++
 1 file changed, 7 insertions(+)


And indeed when I revert commit cfeb6ae8bcb96ccf674724f223661bbcef7b0d0b kernel 6.5.5 succeeds booting with CONFIG_VMAP_STACK=y enabled. dmesg of the successful boot with the reverted commit attached, also kernel .config and the bisect.log.

Regards,
Erhard F.

Download attachment "config_655_g4-van-" of type "application/octet-stream" (103259 bytes)

Download attachment "dmesg_655_g4" of type "application/octet-stream" (35821 bytes)

View attachment "bisect.log" of type "text/x-log" (2838 bytes)

Powered by blists - more mailing lists