Message-ID: <8e0f8371-095b-dab1-8acd-957fbac7046c@amd.com>
Date: Mon, 19 Aug 2024 13:30:00 -0500
From: Tom Lendacky <thomas.lendacky@....com>
To: Ashish Kalra <Ashish.Kalra@....com>, tglx@...utronix.de,
 mingo@...hat.com, bp@...en8.de, dave.hansen@...ux.intel.com, x86@...nel.org,
 hpa@...or.com
Cc: michael.roth@....com, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] x86/sev: ensure that RMP table fixups are reserved for
 memblock

On 8/15/24 17:16, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@....com>
> 
> The BIOS reserves RMP table memory via e820 reservations, but since this
> can still lead to RMP page faults during kexec if the host tries to access
> memory within the same 2MB region, commit 400fea4b9651 ("x86/sev: Add
> callback to apply RMP table fixups for kexec") was added to adjust the
> e820 reservations for the RMP table so that the entire 2MB range at the
> start/end of the RMP table is marked reserved. The e820 reservations are
> then later passed to firmware via SNP_INIT where they get marked HV-Fixed.
> 
> The RMP table fixups are done after the e820 ranges have been added to
> memblock, allowing the fixup ranges to still be allocated and used by
> the system. The problem is that this memory range is now marked reserved
> in the e820 tables and during SNP initialization these reserved ranges
> are made HV-Fixed. This means that the pages cannot be used by an SNP
> guest, only by the hypervisor. However, the memory management subsystem
> does not make this distinction and can allocate one of those pages to an
> SNP guest. This will ultimately result in RMPUPDATE failures associated
> with the guest, causing it to fail to start or terminate when accessing
> the HV-Fixed page.
> 
> The issue is captured below with memblock=debug:
> 
> [    0.000000] SEV-SNP: *** DEBUG: snp_probe_rmptable_info:352 - rmp_base=0x280d4800000, rmp_end=0x28357efffff
> ...
> [    0.000000] BIOS-provided physical RAM map:
> ...
> [    0.000000] BIOS-e820: [mem 0x00000280d4800000-0x0000028357efffff] reserved
> [    0.000000] BIOS-e820: [mem 0x0000028357f00000-0x0000028357ffffff] usable
> ...
> ...
> [    0.183593] memblock add: [0x0000028357f00000-0x0000028357ffffff] e820__memblock_setup+0x74/0xb0
> ...
> [    0.203179] MEMBLOCK configuration:
> [    0.207057]  memory size = 0x0000027d0d194000 reserved size = 0x0000000009ed2c00
> [    0.215299]  memory.cnt  = 0xb
> ...
> [    0.311192]  memory[0x9]     [0x0000028357f00000-0x0000028357ffffff], 0x0000000000100000 bytes flags: 0x0
> ...
> ...
> [    0.419110] SEV-SNP: Reserving start/end of RMP table on a 2MB boundary [0x0000028357e00000]
> [    0.428514] e820: update [mem 0x28357e00000-0x28357ffffff] usable ==> reserved
> [    0.428517] e820: update [mem 0x28357e00000-0x28357ffffff] usable ==> reserved
> [    0.428520] e820: update [mem 0x28357e00000-0x28357ffffff] usable ==> reserved
> ...
> ...
> [    5.604051] MEMBLOCK configuration:
> [    5.607922]  memory size = 0x0000027d0d194000 reserved size = 0x0000000011faae02
> [    5.616163]  memory.cnt  = 0xe
> ...
> [    5.754525]  memory[0xc]     [0x0000028357f00000-0x0000028357ffffff], 0x0000000000100000 bytes on node 0 flags: 0x0
> ...
> ...
> [   10.080295] Early memory node ranges[   10.168065]
> ...
> node   0: [mem 0x0000028357f00000-0x0000028357ffffff]
> ...
> ...
> [ 8149.348948] SEV-SNP: RMPUPDATE failed for PFN 28357f7c, pg_level: 1, ret: 2
> 
> As shown above, the memblock allocations show 1MB after the end of the RMP
> as available for allocation, which is what the RMP table fixups has made
> reserved. This memory range subsequently gets allocated as SNP guest
> memory, resulting in an RMPUPDATE failure.
> 
> This can potentially be fixed by not reserving the memory range in the
> e820 table, but that causes kexec failures when using the
> KEXEC_FILE_LOAD syscall.
> 
> The solution is to use memblock_reserve() to mark the memory reserved
> for the system, ensuring that it cannot be allocated to an SNP guest.
> 
> Since HV-Fixed memory is still readable/writable by the host,
> this only ends up being a problem if the memory in this range requires
> a page state change, which generally will only happen when allocating
> memory in this range to be used for running SNP guests, which is now
> possible with the SNP hypervisor support in kernel 6.11.
> 
> For this reason, this patch is not being marked for stable, even though
> it fixes handling that was introduced in kernel 6.9.
> 
> Fixes: 400fea4b9651 ("x86/sev: Add callback to apply RMP table fixups for kexec")
> Suggested-by: Thomas Lendacky <thomas.lendacky@....com>
> Signed-off-by: Ashish Kalra <ashish.kalra@....com>

Reviewed-by: Tom Lendacky <thomas.lendacky@....com>

> ---
>  arch/x86/virt/svm/sev.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
> index 0ce17766c0e5..9a6a943d8e41 100644
> --- a/arch/x86/virt/svm/sev.c
> +++ b/arch/x86/virt/svm/sev.c
> @@ -173,6 +173,8 @@ static void __init __snp_fixup_e820_tables(u64 pa)
>  		e820__range_update(pa, PMD_SIZE, E820_TYPE_RAM, E820_TYPE_RESERVED);
>  		e820__range_update_table(e820_table_kexec, pa, PMD_SIZE, E820_TYPE_RAM, E820_TYPE_RESERVED);
>  		e820__range_update_table(e820_table_firmware, pa, PMD_SIZE, E820_TYPE_RAM, E820_TYPE_RESERVED);
> +		if (!memblock_is_region_reserved(pa, PMD_SIZE))
> +			memblock_reserve(pa, PMD_SIZE);
>  	}
>  }
>  

Powered by blists - more mailing lists