linux-kernel - Re: KVM guest sometimes failed to boot because of kernel stack overflow if KPTI is enabled on a hisilicon ARM64 platform.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 26 Jun 2018 18:47:46 +0100
From:   Will Deacon <will.deacon@....com>
To:     Wei Xu <xuwei5@...ilicon.com>
Cc:     James Morse <james.morse@....com>, mark.rutland@....com,
        catalin.marinas@....com, Linuxarm <linuxarm@...wei.com>,
        Zhangyi ac <zhangyi.ac@...wei.com>, suzuki.poulose@....com,
        marc.zyngier@....com,
        "Xiongfanggou (James)" <james.xiong@...wei.com>,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org,
        dave.martin@....com,
        "Liyuan (Larry, Turing Solution)" <Larry.T@...wei.com>,
        libeijian@...ilicon.com
Subject: Re: KVM guest sometimes failed to boot because of kernel stack
 overflow if KPTI is enabled on a hisilicon ARM64 platform.

Hi Wei,

On Wed, Jun 27, 2018 at 01:16:44AM +0800, Wei Xu wrote:
> Today I tried the kernel 4.18-rc2(defconfig, no change on top) with qemu
> 2.12.0.
> The guest sometimes still failed to boot. But the crash reason is different.
> Could you please share any hint?
> Thanks!
> 
> The guest boot log is as below:
> ===========================
> 
>     estuary:/$ ./qemu-system-aarch64 -machine virt,kernel_irqchip=on,gic-v
>     ersion=3 -cpu host -enable-kvm -smp 1 -m 1024 -kernel ./Image-4.18-joyx
> -initrd
>     ../mini-rootfs-arm64.cpio.gz -nographic -append "rdinit=init
> console=ttyAMA0 ear
>     lycon=pl011,0x9000000"
> 
>     [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x480fd010]
>     [    0.000000] Linux version 4.18.0-rc2-58583-g7daf201-dirty

I'm still suspicious that this is 4.18-rc2 with "no change on top" ^^^ !

>     [    0.048119] Unable to handle kernel NULL pointer dereference at
> virtual address 0000000000000288
>     [    0.048991] Mem abort info:
>     [    0.049267]   ESR = 0x96000004
>     [    0.049567]   Exception class = DABT (current EL), IL = 32 bits
>     [    0.050146]   SET = 0, FnV = 0
>     [    0.050446]   EA = 0, S1PTW = 0
>     [    0.050754] Data abort info:
>     [    0.051038]   ISV = 0, ISS = 0x00000004
>     [    0.051921]   CM = 0, WnR = 0
>     [    0.054936] [0000000000000288] user address but active_mm is swapper
>     [    0.061427] Internal error: Oops: 96000004 [#1] PREEMPT SMP
>     [    0.067080] Modules linked in:
>     [    0.070206] CPU: 0 PID: 13 Comm: migration/0 Not tainted
> 4.18.0-rc2-58583-g7daf201-dirty #20
>     [    0.078745] Hardware name: linux,dummy-virt (DT)
>     [    0.083433] pstate: 60400085 (nZCv daIf +PAN -UAO)
>     [    0.088258] pc : kpti_install_ng_mappings+0x154/0x214
>     [    0.093319] lr : kpti_install_ng_mappings+0x120/0x214
>     [    0.098483] sp : ffff0000093fbce0
>     [    0.101854] x29: ffff0000093fbce0 x28: ffff000008ee5000
>     [    0.107263] x27: ffff000008ee5000 x26: ffff00000923b000
>     [    0.112568] x25: ffff0000090ac000 x24: ffff0000091d9000
>     [    0.117983] x23: ffff000008ee5000 x22: 00000000411d8000
>     [    0.123392] x21: ffff00000923b000 x20: 0000000000000000
>     [    0.128801] x19: ffff0000091d8000 x18: 000000003455d99d
>     [    0.134209] x17: 0000000000000001 x16: 00f8000040ffff13
>     [    0.139513] x15: 000000007dff5000 x14: 000000007dff5000
>     [    0.144920] x13: 00f800007fe00f11 x12: 000000007dff7000
>     [    0.150329] x11: 000000007dff7000 x10: 0000000000000000
>     [    0.155633] x9 : 000000007dff8000 x8 : 000000007dff8000
>     [    0.161042] x7 : 0000000000000000 x6 : 000000004123c000
>     [    0.166451] x5 : 000000004123c000 x4 : 0000000040a5f3d4
>     [    0.171860] x3 : 0000000000000000 x2 : 000000004123b000
>     [    0.177163] x1 : ffff0000090acd88 x0 : ffff80003ca627c0

So looking at the disassembly, we access idmap_t0sz as part of
cpu_install_idmap() and it looks like we push its page address to the
stack:

>        0xffff000008091ffc <+128>:   adrp    x3, 0xffff000009096000 <early_node_cpu_hwid+1440>

[...]

>        0xffff000008092044 <+200>:   str     x3, [x29,#96]

Then after we've come back from the asm call, we want to access idmap_t0sz
again as part of cpu_uninstall_idmap() so we pop it back off:

>        0xffff0000080920cc <+336>:   ldr     x3, [x29,#96]
>        0xffff0000080920d0 <+340>:   ldr     x0, [x3,#648]

And this access is the one that faults, because we popped off NULL.

So actually, rather than faulting on the stack access, we're managing to
load zeroes from somewhere, so it could still be indicative of page table
corruption for the stack mapping.

If you look at the __idmap_kpti_put_pgtable_ent_ng asm macro, can you try
replacing:

	dc      civac, cur_\()\type\()p

with:

	dc      ivac, cur_\()\type\()p

please? Only do this for the guest kernel, not the host. KVM will upgrade
the clean to a clean+invalidate, so it's interesting to see if this has
an effect on the behaviour.

Will