linux-kernel - Re: [PATCH] arm64: invalidate TLB before turning MMU on

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1544708386.18411.13.camel@lca.pw>
Date:   Thu, 13 Dec 2018 08:39:46 -0500
From:   Qian Cai <cai@....pw>
To:     Bhupesh Sharma <bhsharma@...hat.com>
Cc:     Catalin Marinas <catalin.marinas@....com>,
        Will Deacon <will.deacon@....com>,
        Ard Biesheuvel <ard.biesheuvel@...aro.org>,
        Marc Zyngier <marc.zyngier@....com>,
        kexec mailing list <kexec@...ts.infradead.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        AKASHI Takahiro <takahiro.akashi@...aro.org>,
        James Morse <james.morse@....com>,
        linux-arm-kernel <linux-arm-kernel@...ts.infradead.org>,
        Bhupesh SHARMA <bhupesh.linux@...il.com>
Subject: Re: [PATCH] arm64: invalidate TLB before turning MMU on

On Thu, 2018-12-13 at 11:10 +0530, Bhupesh Sharma wrote:
> Hi Qian Cai,
> 
> On Thu, Dec 13, 2018 at 10:53 AM Qian Cai <cai@....pw> wrote:
> > 
> > On this HPE Apollo 70 arm64 server with 256 CPUs, triggering a crash
> > dump just hung. It has 4 threads on each core. Each 2-core share a same
> > L1 and L2 caches, so that is 8 CPUs shares those. All CPUs share a same
> > L3 cache.
> > 
> > It turned out that this was due to the TLB contained stale entries (or
> > uninitialized junk which just happened to look valid) from the first
> > kernel before turning the MMU on in the second kernel which caused this
> > instruction hung,
> > 
> > msr     sctlr_el1, x0
> > 
> > Signed-off-by: Qian Cai <cai@....pw>
> > ---
> >  arch/arm64/kernel/head.S | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> > index 4471f570a295..5196f3d729de 100644
> > --- a/arch/arm64/kernel/head.S
> > +++ b/arch/arm64/kernel/head.S
> > @@ -771,6 +771,10 @@ ENTRY(__enable_mmu)
> >         msr     ttbr0_el1, x2                   // load TTBR0
> >         msr     ttbr1_el1, x1                   // load TTBR1
> >         isb
> > +       dsb     nshst
> > +       tlbi    vmalle1                         // invalidate TLB
> > +       dsb     nsh
> > +       isb
> 
> This will be executed both for the primary and kdump kernel, right? I
> don't think we really want to invalidate the TLB when booting the
> primary kernel.
> It would be too slow and considering that we need to minimize boot
> timings on embedded arm64 devices, I think it would not be a good
> idea.

Yes, it will be executed for the first kernel as well. As James mentioned, it
needs to be done to invalidate TLB that might be used by bootloader anyway.

> 
> >         msr     sctlr_el1, x0
> >         isb
> >         /*
> > --
> > 2.17.2 (Apple Git-113)
> > 
> 
> Also did you check this issue I reported on the HPE apollo machines
> some days back with the kdump kernel boot
> <https://www.spinics.net/lists/kexec/msg21750.html>.
> Can you please confirm that you are not facing the same issue (as I
> suspect from reading your earlier Bug Report) on the HPE apollo
> machine. Also adding 'earlycon' to the bootargs being passed to the
> kdump kernel you can see if you are able to atleast get some console
> output from the kdump kernel.

No, here did not encounter the problem you mentioned.