linux-kernel - Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <d9063c37-a965-d5cf-e923-c0c9f6ddc044@redhat.com>
Date:   Fri, 11 Dec 2020 22:59:59 +0100
From:   Paolo Bonzini <pbonzini@...hat.com>
To:     Thomas Gleixner <tglx@...utronix.de>,
        Marcelo Tosatti <mtosatti@...hat.com>
Cc:     Maxim Levitsky <mlevitsk@...hat.com>, kvm@...r.kernel.org,
        "H. Peter Anvin" <hpa@...or.com>, Jonathan Corbet <corbet@....net>,
        Jim Mattson <jmattson@...gle.com>,
        Wanpeng Li <wanpengli@...cent.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Vitaly Kuznetsov <vkuznets@...hat.com>,
        Sean Christopherson <sean.j.christopherson@...el.com>,
        open list <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...hat.com>,
        "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" <x86@...nel.org>,
        Joerg Roedel <joro@...tes.org>, Borislav Petkov <bp@...en8.de>,
        Shuah Khan <shuah@...nel.org>,
        Andrew Jones <drjones@...hat.com>,
        Oliver Upton <oupton@...gle.com>,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>
Subject: Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

On 11/12/20 22:04, Thomas Gleixner wrote:
>> Its 100ms off with migration, and can be reduced further (customers
>> complained about 5 seconds but seem happy with 0.1ms).
> What is 100ms? Guaranteed maximum migration time?

I suppose it's the length between the time from KVM_GET_CLOCK and 
KVM_GET_MSR(IA32_TSC) to KVM_SET_CLOCK and KVM_SET_MSR(IA32_TSC).  But 
the VM is paused for much longer, the sequence for the non-live part of 
the migration (aka brownout) is as follows:

     pause
     finish sending RAM            receive RAM               ~1 sec
     send paused-VM state          finish receiving RAM     \
                                   receive paused-VM state   ) 0.1 sec
                                   restart                  /

The nanosecond and TSC times are sent as part of the paused-VM state at 
the very end of the live migration process.

So it's still true that the time advances during live migration 
brownout; 0.1 seconds is just the final part of the live migration 
process.  But for _live_ migration there is no need to design things 
according to "people are happy if their clock is off by 0.1 seconds 
only".  Again, save-to-disk, reverse debugging and the like are a 
different story, which is why KVM should delegate policy to userspace 
(while documenting how to do it right).

Paolo

> CLOCK_REALTIME and CLOCK_TAI are off by the time the VM is paused and
> this state persists up to the point where NTP corrects it with a time
> jump.
> 
> So if migration takes 5 seconds then CLOCK_REALTIME is not off by 100ms
> it's off by 5 seconds.
> 
> CLOCK_MONOTONIC/BOOTTIME might be off by 100ms between pause and resume.
>