linux-kernel - RE: [RFC PATCH 0/2] Avoid booting stall caused by idmap_kpti_install_ng

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <AM6PR08MB4376549A9337A835B68F696DF7BD0@AM6PR08MB4376.eurprd08.prod.outlook.com>
Date:   Mon, 25 Jan 2021 06:36:51 +0000
From:   Justin He <Justin.He@....com>
To:     Marc Zyngier <maz@...nel.org>
CC:     Catalin Marinas <Catalin.Marinas@....com>,
        Will Deacon <will@...nel.org>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Anshuman Khandual <Anshuman.Khandual@....com>,
        Suzuki Poulose <Suzuki.Poulose@....com>,
        Mark Rutland <Mark.Rutland@....com>,
        "Gustavo A. R. Silva" <gustavoars@...nel.org>,
        Richard Henderson <richard.henderson@...aro.org>,
        Dave P Martin <Dave.Martin@....com>,
        Steven Price <Steven.Price@....com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mike Rapoport <rppt@...nel.org>,
        Ard Biesheuvel <ardb@...nel.org>,
        Gavin Shan <gshan@...hat.com>,
        Kefeng Wang <wangkefeng.wang@...wei.com>,
        Mark Brown <broonie@...nel.org>,
        Cristian Marussi <Cristian.Marussi@....com>
Subject: RE: [RFC PATCH 0/2] Avoid booting stall caused by
 idmap_kpti_install_ng_mappings

Hi Marc

> -----Original Message-----
> From: Justin He
> Sent: Wednesday, January 20, 2021 11:56 PM
> To: Marc Zyngier <maz@...nel.org>
> Cc: Catalin Marinas <Catalin.Marinas@....com>; Will Deacon
> <will@...nel.org>; linux-arm-kernel@...ts.infradead.org; linux-
> kernel@...r.kernel.org; Anshuman Khandual <Anshuman.Khandual@....com>;
> Suzuki Poulose <Suzuki.Poulose@....com>; Mark Rutland
> <Mark.Rutland@....com>; Gustavo A. R. Silva <gustavoars@...nel.org>;
> Richard Henderson <richard.henderson@...aro.org>; Dave P Martin
> <Dave.Martin@....com>; Steven Price <Steven.Price@....com>; Andrew Morton
> <akpm@...ux-foundation.org>; Mike Rapoport <rppt@...nel.org>; Ard
> Biesheuvel <ardb@...nel.org>; Gavin Shan <gshan@...hat.com>; Kefeng Wang
> <wangkefeng.wang@...wei.com>; Mark Brown <broonie@...nel.org>; Cristian
> Marussi <Cristian.Marussi@....com>
> Subject: RE: [RFC PATCH 0/2] Avoid booting stall caused by
> idmap_kpti_install_ng_mappings
> 
> Hi Marc
> 
> > -----Original Message-----
> > From: Marc Zyngier <maz@...nel.org>
> > Sent: Wednesday, January 20, 2021 6:58 PM
> > To: Justin He <Justin.He@....com>
> > Cc: Catalin Marinas <Catalin.Marinas@....com>; Will Deacon
> > <will@...nel.org>; linux-arm-kernel@...ts.infradead.org; linux-
> > kernel@...r.kernel.org; Anshuman Khandual <Anshuman.Khandual@....com>;
> > Suzuki Poulose <Suzuki.Poulose@....com>; Mark Rutland
> > <Mark.Rutland@....com>; Gustavo A. R. Silva <gustavoars@...nel.org>;
> > Richard Henderson <richard.henderson@...aro.org>; Dave P Martin
> > <Dave.Martin@....com>; Steven Price <Steven.Price@....com>; Andrew Morton
> > <akpm@...ux-foundation.org>; Mike Rapoport <rppt@...nel.org>; Ard
> > Biesheuvel <ardb@...nel.org>; Gavin Shan <gshan@...hat.com>; Kefeng Wang
> > <wangkefeng.wang@...wei.com>; Mark Brown <broonie@...nel.org>; Cristian
> > Marussi <Cristian.Marussi@....com>
> > Subject: Re: [RFC PATCH 0/2] Avoid booting stall caused by
> > idmap_kpti_install_ng_mappings
> >
> > Hi Justin,
> >
> > On 2021-01-20 04:51, Justin He wrote:
> > > Hi,
> > > Kindly ping 😊
> > >
> > >> -----Original Message-----
> > >> From: Jia He <justin.he@....com>
> > >> Sent: Wednesday, January 13, 2021 9:41 AM
> > >> To: Catalin Marinas <Catalin.Marinas@....com>; Will Deacon
> > >> <will@...nel.org>; linux-arm-kernel@...ts.infradead.org; linux-
> > >> kernel@...r.kernel.org
> > >> Cc: Anshuman Khandual <Anshuman.Khandual@....com>; Suzuki Poulose
> > >> <Suzuki.Poulose@....com>; Justin He <Justin.He@....com>; Mark Rutland
> > >> <Mark.Rutland@....com>; Gustavo A. R. Silva <gustavoars@...nel.org>;
> > >> Richard Henderson <richard.henderson@...aro.org>; Dave P Martin
> > >> <Dave.Martin@....com>; Steven Price <Steven.Price@....com>; Andrew
> > >> Morton
> > >> <akpm@...ux-foundation.org>; Mike Rapoport <rppt@...nel.org>; Ard
> > >> Biesheuvel <ardb@...nel.org>; Gavin Shan <gshan@...hat.com>; Kefeng
> > >> Wang
> > >> <wangkefeng.wang@...wei.com>; Mark Brown <broonie@...nel.org>; Marc
> > >> Zyngier
> > >> <maz@...nel.org>; Cristian Marussi <Cristian.Marussi@....com>
> > >> Subject: [RFC PATCH 0/2] Avoid booting stall caused by
> > >>
> > >> There is a 10s stall in idmap_kpti_install_ng_mappings when kernel
> > >> boots
> > >> on a Ampere EMAG server.
> > >>
> > >> Commit f992b4dfd58b ("arm64: kpti: Add ->enable callback to remap
> > >> swapper using nG mappings") updates the nG bit runtime if kpti is
> > >> required.
> > >>
> > >> But things get worse if rodata=full in map_mem(). NO_BLOCK_MAPPINGS |
> > >> NO_CONT_MAPPINGS is required when creating pagetable mapping. Hence
> > >> all
> > >> ptes are fully mapped in this case. On a Ampere EMAG server with 256G
> > >> memory(pagesize=4k), it causes the 10s stall.
> > >>
> > >> After moving init_cpu_features() ahead of early_fixmap_init(), we can
> > >> use
> > >> cpu_have_const_cap earlier than before. Hence we can avoid this stall
> > >> by updating arm64_use_ng_mappings.
> > >>
> > >> After this patch series, it reduces the kernel boot time from 14.7s to
> > >> 4.1s:
> > >> Before:
> > >> [   14.757569] Freeing initrd memory: 60752K
> > >> After:
> > >> [    4.138819] Freeing initrd memory: 60752K
> > >>
> > >> Set it as RFC because I want to resolve any other points which I have
> > >> misconerned.
> >
> > But you don't really explain *why* having the CPU Feature discovery
> > early helps at all. Is that so that you can bypass the idmap mapping?
> 
> Adding nG bits can be avoided by having the discovery of boot cpu feature
> earlier since the nG bit had been set in PTE_MAYBE_NG/PMD_MAYBE_NG
> 
> Before this patch:
> 1. kernel will firstly create mapping in setup_arch->paging_init->map_mem
> -> __map_memblock
> 2. Then if kpti is required, kernel will add nG bits for each pte entry.
> 3. In extreme case, e.g. physical memory is 256G,rodata=full, and pagesize
> is 4K, the nG bits updating in step 2 takes about 10s.
> 
> > I'd expect something that explain the problem instead of paraphrasing
> > the patches.
> >
> > Another thing is whether you have tested this on some ThunderX HW
> 
> I will find a TX1 as you told to see any difference.
> 
> 
I fortunately found a cavium TX1. 
Seems that unmap_kernel_at_el0 is false:
...
[    0.000000] Machine model: Cavium ThunderX CN88XX board
...
[    0.000000] CPU features: kernel page table isolation forced OFF by ARM64_WORKAROUND_CAVIUM_27456
...

Hence no such stall *before* and *after* this patch set because kpti is not enabled.


--
Cheers,
Justin (Jia He)