linux-kernel - Re: [Regression v5.19-rc1] kernel fails to boot, no console output (Re: [PATCH v12 38/46] x86/sev: Add SEV-SNP feature detection/setup)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 17 Aug 2022 10:40:40 +0200
From:   Jeremi Piotrowski <jpiotrowski@...ux.microsoft.com>
To:     Michael Roth <michael.roth@....com>
Cc:     NOMURA JUNICHI(野村　淳一) 
        <junichi.nomura@....com>, LKML <linux-kernel@...r.kernel.org>,
        "x86@...nel.org" <x86@...nel.org>, "bp@...e.de" <bp@...e.de>
Subject: Re: [Regression v5.19-rc1] kernel fails to boot, no console output
 (Re: [PATCH v12 38/46] x86/sev: Add SEV-SNP feature detection/setup)

On Tue, Aug 16, 2022 at 10:06:37AM -0500, Michael Roth wrote:
> On Tue, Aug 16, 2022 at 04:25:30PM +0200, Jeremi Piotrowski wrote:
> > On Fri, Jun 24, 2022 at 12:44:52AM +0000, NOMURA JUNICHI(野村 淳一) wrote:
> > > I found crash kexec fails to boot the 2nd kernel since v5.19-rc1 and
> > > git bisect points to this as a bad commit:
> > > 
> > >   commit b190a043c49af4587f5e157053f909192820522a
> > >   Author: Michael Roth <michael.roth@....com>
> > >   Date:   Thu Feb 24 10:56:18 2022 -0600
> > > 
> > >     x86/sev: Add SEV-SNP feature detection/setup
> > > 
> > >     Initial/preliminary detection of SEV-SNP is done via the Confidential
> > >     Computing blob. Check for it prior to the normal SEV/SME feature
> > >     initialization, and add some sanity checks to confirm it agrees with
> > >     SEV-SNP CPUID/MSR bits.
> > > 
> > 
> > Hi Michael,
> > 
> > I too have bisected an issue to this commit and my issue is not fixed by
> > b57feed2cc2622ae14b2fa62f19e973e5e0a60cf. I have a Dell R6515 with AMD EPYC
> > 7513 that is supposed to act as an SNP host, but after v5.19-rc1 I don't get
> > any console output and the machine resets shortly after grub loads the
> > kernel.
> > 
> > The bisect was done on 5.18+SNP patches that were merged into 5.19-rc1, this
> > is the full patch list:
> > 
> >   git log --oneline v5.18-rc1..eb39e37d5cebdf0f63ee2a315fc23b035d81b4b0^2
> > 
> > If I comment out the following lines, the machine boots correctly and if I also
> > have the SNP host patches applied SNP guests work correctly. Applying
> 
> Hi Jeremi,
> 
> Can you retry with earlyprintk enabled via kernel cmdline? I'm trying to see
> if you're getting these messages and making it past the boot/compressed kernel
> (which has its own snp_init()):
> 
>   EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path^M
>   early console in extract_kernel^M
>   input_data: 0x00000000352fd4cc^M
>   input_len: 0x0000000000c5adcb^M
>   output: 0x0000000033000000^M
>   output_len: 0x0000000002f07ec4^M
>   kernel_total_size: 0x000000000242c000^M
>   needed_size: 0x0000000003000000^M
>   trampoline_32bit: 0x000000000009d000^M
>   Physical KASLR using RDRAND RDTSC...^M
>   Virtual KASLR using RDRAND RDTSC...^M
>   ^M
>   Decompressing Linux... Parsing ELF... Performing relocations... done.^M
>   Booting the kernel.
>   [    0.000000] Linux version ...
> 

Ok, I was missing CONFIG_X86_VERBOSE_BOOTUP=y. With that added, when the kernel
doesn't boot this is the output:

  Loading Linux 5.18.0-snp-host-dev+ ...
  Loading initial ramdisk ...
  early console in extract_kernel
  input_data: 0x00000000068824cc
  input_len: 0x000000000092cb45
  output: 0x0000000004200000
  output_len: 0x0000000002f5eff4
  kernel_total_size: 0x0000000002430000
  needed_size: 0x0000000003000000
  trampoline_32bit: 0x000000000009d000
  Physical KASLR using RDRAND RDTSC...
  Virtual KASLR using RDRAND RDTSC...
  
  Decompressing Linux... Parsing ELF... Performing relocations... done.
  Booting the kernel.

> Also, instead of commenting out snp_init() below, can you retry with this
> change instead?
> 
>   diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
>   index 7b668f91c9ab..e9e55a99d60f 100644
>   --- a/arch/x86/kernel/sev.c
>   +++ b/arch/x86/kernel/sev.c
>   @@ -2083,9 +2083,12 @@ static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp)
>            * boot/decompression kernel, the CC blob may have been passed via
>            * setup_data instead.
>            */
>   +#if 0
>           cc_info = find_cc_blob_setup_data(bp);
>           if (!cc_info)
>                   return NULL;
>   +#endif
>   +       return NULL;
>   
>    found_cc_info:
>           if (cc_info->magic != CC_BLOB_SEV_HDR_MAGIC)
> 

This version does not boot.

> And if that does not avoid the issue, can you retry with the below change
> (without the above change in place)?
> 
>   diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
>   index 7b668f91c9ab..0b430f98e053 100644
>   --- a/arch/x86/kernel/sev.c
>   +++ b/arch/x86/kernel/sev.c
>   @@ -2073,10 +2073,12 @@ static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp)
>           struct cc_blob_sev_info *cc_info;
>   
>           /* Boot kernel would have passed the CC blob via boot_params. */
>   +#if 0
>           if (bp->cc_blob_address) {
>                   cc_info = (struct cc_blob_sev_info *)(unsigned long)bp->cc_blob_address;
>                   goto found_cc_info;
>           }
>   +#endif
>   
>           /*
>            * If kernel was booted directly, without the use of the
>   @@ -2087,7 +2089,7 @@ static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp)
>           if (!cc_info)
>                   return NULL;
>   
>   -found_cc_info:
>   +//found_cc_info:
>           if (cc_info->magic != CC_BLOB_SEV_HDR_MAGIC)
>                   snp_abort();
> 
> Trying to see if you're hitting a separate issue or not.

This version boots correctly.

> 
> Thanks,
> 
> Mike
> 
>