linux-kernel - Re: Does earlyprintk=ttyS0 work for an AMD SNP guest on KVM?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230216091431.GA10166@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
Date:   Thu, 16 Feb 2023 01:14:31 -0800
From:   Jeremi Piotrowski <jpiotrowski@...ux.microsoft.com>
To:     Dexuan Cui <decui@...rosoft.com>
Cc:     Tom Lendacky <thomas.lendacky@....com>,
        Borislav Petkov <bp@...en8.de>,
        "sandipan.das@....com" <sandipan.das@....com>,
        "Gupta, Pankaj" <pankaj.gupta@....com>,
        "ray.huang@....com" <ray.huang@....com>,
        "brijesh.singh@....com" <brijesh.singh@....com>,
        "michael.roth@....com" <michael.roth@....com>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "x86@...nel.org" <x86@...nel.org>,
        Tianyu Lan <Tianyu.Lan@...rosoft.com>,
        "linux-hyperv@...r.kernel.org" <linux-hyperv@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: Does earlyprintk=ttyS0 work for an AMD SNP guest on KVM?

On Thu, Feb 16, 2023 at 04:40:14AM +0000, Dexuan Cui wrote:
> Hi all,
> With the earlyprintk=ttyS0 kernel parameter, a C-bit mode Linux SNP guest
> on Hyper-V always decides to crash via sev_es_terminate() in
> do_boot_stage2_vc(), because early_setup_ghcb() fails:
> 
> early_setup_ghcb() ->
>   set_page_decrypted() ->
>     set_clr_page_flags() ->
>       split_large_pmd() ->
>         alloc_pgt_page() fails to allocate memory.
> 
> static void *alloc_pgt_page(void *context)
> {
> ...
>         /* Validate there is space available for a new page. */
>         if (pages->pgt_buf_offset >= pages->pgt_buf_size) {
>   ...
>               return NULL;
>         }
> ...
> }
> 
> alloc_pgt_page() fails to allocate memory because both
> pages->pgt_buf_offset and pages->pgt_buf_size are zero.
> 
> 
> pgt_data.pgt_buf_size is zero because of this line in
> initialize_identity_maps()
>      pgt_data.pgt_buf_size = BOOT_PGT_SIZE - BOOT_INIT_PGT_SIZE;
> 
> void initialize_identity_maps(void *rmode)
> {
> ...
>         top_level_pgt = read_cr3_pa();
>         if (p4d_offset((pgd_t *)top_level_pgt, 0) == (p4d_t *)_pgtable) {
>                 pgt_data.pgt_buf = _pgtable + BOOT_INIT_PGT_SIZE;
>                 pgt_data.pgt_buf_size = BOOT_PGT_SIZE - BOOT_INIT_PGT_SIZE;
>                 memset(pgt_data.pgt_buf, 0, pgt_data.pgt_buf_size);
>         } else {
>                 pgt_data.pgt_buf = _pgtable;
>                 pgt_data.pgt_buf_size = BOOT_PGT_SIZE;
>                 memset(pgt_data.pgt_buf, 0, pgt_data.pgt_buf_size);
>                 top_level_pgt = (unsigned long)alloc_pgt_page(&pgt_data);

I just tested an SNP guest on KVM with and without CONFIG_RANDOMIZE_BASE.
In both cases we end up in the else() branch.
With CONFIG_RANDOMIZE_BASE BOOT_PGT_SIZE=0x13000
Without CONFIG_RANDOMMIZE_BASE BOOT_PGT_SIZE=0x6000.

So in both cases pgt_data.pgt_buf_size != 0. 

Getting into that first branch would require having 5-level paging supported
(CONFIG_X86_5LEVEL=y) and enabled inside the guest, I don't have that on any
hardware I have access to.

Jeremi

>         }
> 
> In arch/x86/include/asm/boot.h, BOOT_PGT_SIZE equals
> BOOT_INIT_PGT_SIZE if CONFIG_RANDOMIZE_BASE is not defined 
> (which is my case):
>  
> # define BOOT_INIT_PGT_SIZE     (6*4096)
> 
> # ifdef CONFIG_RANDOMIZE_BASE
> ...
> #  ifdef CONFIG_X86_VERBOSE_BOOTUP
> #   define BOOT_PGT_SIZE        (19*4096)
> #  else /* !CONFIG_X86_VERBOSE_BOOTUP */
> #   define BOOT_PGT_SIZE        (17*4096)
> #  endif
> # else /* !CONFIG_RANDOMIZE_BASE */
> #  define BOOT_PGT_SIZE         BOOT_INIT_PGT_SIZE
> # endif
> 
> I think this means: if CONFIG_RANDOMIZE_BASE is not defined,
> earlyprintk=ttyS0 also doesn't work for an SNP guest on KVM? 
> Sorry I don't have a KVM environment at hand to test it.
> 
> If I define CONFIG_RANDOMIZE_BASE, my C-bit mode SNP guest crashes
> even ealier -- it looks like CONFIG_RANDOMIZE_BASE is incompatible
> with my guest on Hyper-V due to some reason I don't know.
> 
> Do you always use CONFIG_RANDOMIZE_BASE for a SNP guest on KVM
> and does earlyprintk=ttyS0 work for you?
> 
> Can you please share your thoughts? Thanks!
> 
> Thanks,
> -- Dexuan