linux-kernel - Re: [PATCHv3 2/2] kvm: x86: implement KVM PM-notifier

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YL6+NysuqhORZEs1@google.com>
Date:   Tue, 8 Jun 2021 09:47:51 +0900
From:   Sergey Senozhatsky <senozhatsky@...omium.org>
To:     Maxim Levitsky <mlevitsk@...hat.com>
Cc:     Sergey Senozhatsky <senozhatsky@...omium.org>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Marc Zyngier <maz@...nel.org>,
        Vitaly Kuznetsov <vkuznets@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Suleiman Souhlal <suleiman@...gle.com>, x86@...nel.org,
        kvm@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCHv3 2/2] kvm: x86: implement KVM PM-notifier

On (21/06/07 15:54), Maxim Levitsky wrote:
[..]
> Also I would like to add my .02 cents on my observations on what happens when I suspend my system
> with guests running, which I do once in a while.
> I haven't dug deep into it yet as host suspend with VM running wasn't high on my priority list.
>  
> First of all after a host suspend/resume cycle (and that is true on all 3 machines I own),
> the host TSC is reset to 0 on all CPUs, thus while it is still synchronized, it jumps backward.
>  
> Host kernel has no issues coping with this.
>  
> Guests however complain about clocksource watchdog and mark the tsc clocksource as unstable,
> at least when invtsc is used (regardless of this patch, I wasn't able to notice a difference
> with and without it yet).
>  
>  
> [  287.515864] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc' as unstable because the skew is too large:
> [  287.516926] clocksource:                       'kvm-clock' wd_now: 4437767926 wd_last: 429c3c42f5 mask: ffffffffffffffff
> [  287.527100] clocksource:                       'tsc' cs_now: c33f6ce157 cs_last: c1be2ad19f mask: ffffffffffffffff
> [  287.528493] tsc: Marking TSC unstable due to clocksource watchdog
> [  287.556640] clocksource: Switched to clocksource kvm-clock
>  
>  
> This is from Intel system with stable TSC, but I have seen this on my AMD systems as well,
> but these have other issues which might affect this (see below).
>  
> AFAIK, we have code in kvm_arch_hardware_enable for this exact case but it might not work
> correctly or be not enough to deal with this.
>  
> Also I notice that this code sets kvm->arch.backwards_tsc_observed = true which 
> in turn disables the master clock which is not good as well.
>  
> I haven't yet allocated time to investigate this.
>  
>  
> Another bit of information which I didn't start a discussion (but I think I should), 
> which is relevant to AMD systems, is in 'unsynchronized_tsc' function.
> 
> On AMD guest it will mark the TSC as unstable in the guest as long as invtsc is not used.
> I patched that code out for myself, that is why I am mentioning it.

If you are going to fork this discussion the could you please Cc Suleiman
and me? I believe we are having a bunch of problems with the guest clocks
here on our side.