linux-kernel - Re: [PATCH] x86/kvmclock: Stop kvmclocks for hibernate restore

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87eeg2cbm8.fsf@vitty.brq.redhat.com>
Date:   Fri, 26 Mar 2021 14:01:35 +0100
From:   Vitaly Kuznetsov <vkuznets@...hat.com>
To:     Lenny Szubowicz <lszubowi@...hat.com>
Cc:     pbonzini@...hat.com, seanjc@...gle.com, wanpengli@...cent.com,
        jmattson@...gle.com, joro@...tes.org, tglx@...utronix.de,
        mingo@...hat.com, bp@...en8.de, x86@...nel.org, hpa@...or.com,
        kvm@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] x86/kvmclock: Stop kvmclocks for hibernate restore

Vitaly Kuznetsov <vkuznets@...hat.com> writes:

..

> (this is with your v2 included). There's nothing about CPU0 for
> e.g. async PF + timestamps are really interesting. Seems we have issues
> to fix) I'm playing with it right now.

What if we do the following (instead of your patch):

diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 78bb0fae3982..c32392d6329d 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -26,6 +26,7 @@
 #include <linux/kprobes.h>
 #include <linux/nmi.h>
 #include <linux/swait.h>
+#include <linux/syscore_ops.h>
 #include <asm/timer.h>
 #include <asm/cpu.h>
 #include <asm/traps.h>
@@ -598,17 +599,21 @@ static void kvm_guest_cpu_offline(void)
 
 static int kvm_cpu_online(unsigned int cpu)
 {
-	local_irq_disable();
+	unsigned long flags;
+
+	local_irq_save(flags);
 	kvm_guest_cpu_init();
-	local_irq_enable();
+	local_irq_restore(flags);
 	return 0;
 }
 
 static int kvm_cpu_down_prepare(unsigned int cpu)
 {
-	local_irq_disable();
+	unsigned long flags;
+
+	local_irq_save(flags);
 	kvm_guest_cpu_offline();
-	local_irq_enable();
+	local_irq_restore(flags);
 	return 0;
 }
 #endif
@@ -639,6 +644,23 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask,
 	native_flush_tlb_others(flushmask, info);
 }
 
+static int kvm_suspend(void)
+{
+	kvm_guest_cpu_offline();
+
+	return 0;
+}
+
+static void kvm_resume(void)
+{
+	kvm_cpu_online(raw_smp_processor_id());
+}
+
+static struct syscore_ops kvm_syscore_ops = {
+	.suspend	= kvm_suspend,
+	.resume		= kvm_resume,
+};
+
 static void __init kvm_guest_init(void)
 {
 	int i;
@@ -681,6 +703,8 @@ static void __init kvm_guest_init(void)
 	kvm_guest_cpu_init();
 #endif
 
+	register_syscore_ops(&kvm_syscore_ops);
+
 	/*
 	 * Hard lockup detection is enabled by default. Disable it, as guests
 	 * can get false positives too easily, for example if the host is
-- 
2.30.2

This seems to work fine (according to the log, I haven't checked yet
that PV features are actually working):

[   20.678081] PM: hibernation: Creating image:
[   20.689925] PM: hibernation: Need to copy 82048 pages
[    2.302411] kvm-clock: cpu 0, msr 2c201001, primary cpu clock, resume
[    2.302487] PM: Restoring platform NVS memory
[    2.302498] kvm-guest: KVM setup async PF for cpu 0
[    2.302502] kvm-guest: stealtime: cpu 0, msr 3ec2c080
[    2.304754] Enabling non-boot CPUs ...
[    2.304823] x86: Booting SMP configuration:
[    2.304824] smpboot: Booting Node 0 Processor 1 APIC 0x1
[    2.304952] kvm-clock: cpu 1, msr 2c201041, secondary cpu clock
[    2.305400] kvm-guest: KVM setup async PF for cpu 1
[    2.305405] kvm-guest: stealtime: cpu 1, msr 3ecac080
[    2.305786] CPU1 is up
[    2.305818] smpboot: Booting Node 0 Processor 2 APIC 0x2
[    2.305920] kvm-clock: cpu 2, msr 2c201081, secondary cpu clock
[    2.306325] kvm-guest: KVM setup async PF for cpu 2
[    2.306330] kvm-guest: stealtime: cpu 2, msr 3ed2c080
[    2.306599] CPU2 is up
[    2.306627] smpboot: Booting Node 0 Processor 3 APIC 0x3
[    2.306729] kvm-clock: cpu 3, msr 2c2010c1, secondary cpu clock
[    2.307092] kvm-guest: KVM setup async PF for cpu 3
[    2.307097] kvm-guest: stealtime: cpu 3, msr 3edac080
[    2.307383] CPU3 is up
[    2.307560] ACPI: Waking up from system sleep state S4
[    2.318778] sd 0:0:0:0: [sda] Starting disk
[    2.342335] OOM killer enabled.
[    2.342817] Restarting tasks ... done.
[    2.346209] PM: hibernation: hibernation exit

-- 
Vitaly