Message-ID: <82D7661F83C1A047AF7DC287873BF1E172BDF8D0@SHSMSX101.ccr.corp.intel.com>
Date:   Mon, 5 Nov 2018 02:05:28 +0000
From:   "Kang, Luwei" <luwei.kang@...el.com>
To:     Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "x86@...nel.org" <x86@...nel.org>
CC:     "tglx@...utronix.de" <tglx@...utronix.de>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "bp@...en8.de" <bp@...en8.de>, "hpa@...or.com" <hpa@...or.com>,
        "rkrcmar@...hat.com" <rkrcmar@...hat.com>,
        "joro@...tes.org" <joro@...tes.org>,
        "songliubraving@...com" <songliubraving@...com>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "kstewart@...uxfoundation.org" <kstewart@...uxfoundation.org>,
        "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>,
        "thomas.lendacky@....com" <thomas.lendacky@....com>,
        "konrad.wilk@...cle.com" <konrad.wilk@...cle.com>,
        "mattst88@...il.com" <mattst88@...il.com>,
        "Janakarajan.Natarajan@....com" <Janakarajan.Natarajan@....com>,
        "dwmw@...zon.co.uk" <dwmw@...zon.co.uk>,
        "jpoimboe@...hat.com" <jpoimboe@...hat.com>,
        "marcorr@...gle.com" <marcorr@...gle.com>,
        "ubizjak@...il.com" <ubizjak@...il.com>,
        "Christopherson, Sean J" <sean.j.christopherson@...el.com>,
        "jmattson@...gle.com" <jmattson@...gle.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Chao Peng <chao.p.peng@...ux.intel.com>,
        "luwei.k.kang@...il.com" <luwei.k.kang@...il.com>
Subject: RE: [Resend PATCH v13 08/12] KVM: x86: Add Intel PT context switch
 for each vcpu

> >> If you "have to enable or disable anything" it means you have to
> >> override the default.  But the default in this patches is "no change
> >> compared to before the patches", leaving tracing of both host and
> >> guest entirely to the host, so I don't understand your remark.  What
> >> workflow is broken?
> >>
> >>> There already are controls in perf that enable/disable guest tracing.
> >>
> >> You are confusing "tracing guest from the host" and "the guest can
> >> trace itself".  This patchset is adding support for the latter, and
> >> that
> 
> I'm not confusing anything. In the terminology that you're using, the latter breaks the former. This cannot happen.
> 
> >> affects directly whether the tracing CPUID leaf can be added to the
> >> guest.  Therefore it's not perf that can decide whether to turn it
> >> on; KVM must know it when /dev/kvm is opened, which is why it is a
> >> module parameter.
> 
> There is a control in the perf event attribute that enables tracing the guest. If this control is enabled, the kvm needs to stay away from any
> PT related MSRs. Conversely, if kvm is using PT (or, as you say, "the guest is tracing itself"), the host should not be allowed to ask for tracing
> the guest at the same time.

I think what you mentioned "perf event attribute" is "struct perf_event_attr -> exclude_host/exclude_guest" parameter.
Parameter "exclude_host" can use for vPMU in pmc_reprogram_counter() to make the counter disabled before VM-exit; Parameter "exclude_guest" can use for PMU in host to make the counter not include the value in Guest;
For the implementation of vPMU, there have some counters on each logical CPU, and host and guest can using different counter at same time, Not make Guest "stay from" PMU. Is that right? I think this "perf event attribute" not fit for Intel PT.

Intel PT is different, there just have one serials of MSRs in each logical CPU. So this hardware just can be used by host OR guest. In Host-Guest mode, PT feature will be exposed to guest and guest detect this feature certainly can use it like native any way. 
"If this control is enabled, the kvm needs to stay away from any PT related MSRs."
If we pay for a ICL virtual machine support Intel PT from Cloud Vendor but can't be used. Cloud vendor say that Host (or other virtual machine) is using this feature so you can't use it. Why? 

Currently, Intel SDM support three mode of tracing (System-Wide Tracing, Host-Only Tracing and Guest-Only Tracing). below is copy from SDM:
System-Wide Tracing (35.5.2.1): When a host or VMM configures Intel PT to collect trace packets of the entire system, it can leave the relevant VMX controls clear to allow VMX-specific packets to provide information across VMX transitions. 
Host-Only Tracing (35.5.2.2): Trace packets in VMX non-root operation are not desired, the VMM can use the VM-entry MSR-load area to load IA32_RTIT_CTL (clearing TraceEn) to disable trace-packet generation in guests, and use the VM-exit MSR-load area to load IA32_RTIT_CTL to set TraceEn.
Guest-Only Tracing (35.5.2.3): A VMM can configure trace-packet generation while in VMX non-root operation for guests executing normally. 

In Host mode we need to disable PT before VM-entry (no matter if PT is support/enabled or not in guest) and Load Guest PT status if PT is supported in Guest. The Host-Guest mode in this patch set just combined the Host-Only mode and Guest-Only mode to Host-Guest mode because there don't have conflict in these two mode.

As for Host PT will be disable before VM-entry and Host can't aware this behavior. I think this is use case limitation of different working mode and please EXPECT this happened. PT MUST be disabled before VM-entry and switch to Guest PT status in Host-Guest mode (following the description in SDM). Or please use default System-Wide mode.

Thanks,
Luwei Kang

Powered by blists - more mailing lists