linux-kernel - Re: [PATCH v2 2/2] xen/pci: Use APIC directly when APIC virtualization is supported by hardware

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <544E95D5.1020000@oracle.com>
Date:	Mon, 27 Oct 2014 14:58:29 -0400
From:	Boris Ostrovsky <boris.ostrovsky@...cle.com>
To:	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
CC:	david.vrabel@...rix.com, stefano.stabellini@...citrix.com,
	jun.nakajima@...el.com, linux-kernel@...r.kernel.org,
	xen-devel@...ts.xen.org
Subject: Re: [PATCH v2 2/2] xen/pci: Use APIC directly when APIC virtualization
 is supported by hardware

On 10/27/2014 02:39 PM, Konrad Rzeszutek Wilk wrote:
> On Mon, Oct 27, 2014 at 12:05:53PM -0400, Boris Ostrovsky wrote:
>> When hardware supports APIC/x2APIC virtualization we don't need to use pirqs
>> for MSI handling and instead use APIC since most APIC accesses (MMIO or MSR)
>> will now be processed without VMEXITs.
>>
>> As an example, netperf on the original code produces this profile
>> (collected wih 'xentrace -e 0x0008ffff -T 5'):
>>
>>      342 cpu_change
>>      260 CPUID
>>    34638 HLT
>>    64067 INJ_VIRQ
>>    28374 INTR
>>    82733 INTR_WINDOW
>>       10 NPF
>>    24337 TRAP
>>   370610 vlapic_accept_pic_intr
>>   307528 VMENTRY
>>   307527 VMEXIT
>>   140998 VMMCALL
>>      127 wrap_buffer
>>
>> After applying this patch the same test shows
>>
>>      230 cpu_change
>>      260 CPUID
>>    36542 HLT
>>      174 INJ_VIRQ
>>    27250 INTR
>>      222 INTR_WINDOW
>>       20 NPF
>>    24999 TRAP
>>   381812 vlapic_accept_pic_intr
>>   166480 VMENTRY
>>   166479 VMEXIT
>>    77208 VMMCALL
>>       81 wrap_buffer
>>
>> ApacheBench results (ab -n 10000 -c 200) improve by about 10%
> What type of NIC did you use for this?

Intel I350 card, igb driver. I probably should have mentioned this in 
the commit message.

-boris


>
> Thanks!
>> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@...cle.com>
>> ---
>>   arch/x86/include/asm/xen/cpuid.h |   91 ++++++++++++++++++++++++++++++++++++++
>>   arch/x86/pci/xen.c               |    9 ++++
>>   2 files changed, 100 insertions(+), 0 deletions(-)
>>   create mode 100644 arch/x86/include/asm/xen/cpuid.h
>>
>> diff --git a/arch/x86/include/asm/xen/cpuid.h b/arch/x86/include/asm/xen/cpuid.h
>> new file mode 100644
>> index 0000000..0d809e9
>> --- /dev/null
>> +++ b/arch/x86/include/asm/xen/cpuid.h
>> @@ -0,0 +1,91 @@
>> +/******************************************************************************
>> + * arch-x86/cpuid.h
>> + *
>> + * CPUID interface to Xen.
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to
>> + * deal in the Software without restriction, including without limitation the
>> + * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
>> + * sell copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
>> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
>> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>> + * DEALINGS IN THE SOFTWARE.
>> + *
>> + * Copyright (c) 2007 Citrix Systems, Inc.
>> + *
>> + * Authors:
>> + *    Keir Fraser <keir@....org>
>> + */
>> +
>> +#ifndef __XEN_PUBLIC_ARCH_X86_CPUID_H__
>> +#define __XEN_PUBLIC_ARCH_X86_CPUID_H__
>> +
>> +/*
>> + * For compatibility with other hypervisor interfaces, the Xen cpuid leaves
>> + * can be found at the first otherwise unused 0x100 aligned boundary starting
>> + * from 0x40000000.
>> + *
>> + * e.g If viridian extensions are enabled for an HVM domain, the Xen cpuid
>> + * leaves will start at 0x40000100
>> + */
>> +
>> +#define XEN_CPUID_FIRST_LEAF 0x40000000
>> +#define XEN_CPUID_LEAF(i)    (XEN_CPUID_FIRST_LEAF + (i))
>> +
>> +/*
>> + * Leaf 1 (0x40000x00)
>> + * EAX: Largest Xen-information leaf. All leaves up to an including @EAX
>> + *      are supported by the Xen host.
>> + * EBX-EDX: "XenVMMXenVMM" signature, allowing positive identification
>> + *      of a Xen host.
>> + */
>> +#define XEN_CPUID_SIGNATURE_EBX 0x566e6558 /* "XenV" */
>> +#define XEN_CPUID_SIGNATURE_ECX 0x65584d4d /* "MMXe" */
>> +#define XEN_CPUID_SIGNATURE_EDX 0x4d4d566e /* "nVMM" */
>> +
>> +/*
>> + * Leaf 2 (0x40000x01)
>> + * EAX[31:16]: Xen major version.
>> + * EAX[15: 0]: Xen minor version.
>> + * EBX-EDX: Reserved (currently all zeroes).
>> + */
>> +
>> +/*
>> + * Leaf 3 (0x40000x02)
>> + * EAX: Number of hypercall transfer pages. This register is always guaranteed
>> + *      to specify one hypercall page.
>> + * EBX: Base address of Xen-specific MSRs.
>> + * ECX: Features 1. Unused bits are set to zero.
>> + * EDX: Features 2. Unused bits are set to zero.
>> + */
>> +
>> +/* Does the host support MMU_PT_UPDATE_PRESERVE_AD for this guest? */
>> +#define _XEN_CPUID_FEAT1_MMU_PT_UPDATE_PRESERVE_AD 0
>> +#define XEN_CPUID_FEAT1_MMU_PT_UPDATE_PRESERVE_AD  (1u<<0)
>> +
>> +/*
>> + * Leaf 5 (0x40000x04)
>> + * HVM-specific features
>> + */
>> +
>> +/* EAX Features */
>> +/* Virtualized APIC registers */
>> +#define XEN_HVM_CPUID_APIC_ACCESS_VIRT (1u << 0)
>> +/* Virtualized x2APIC accesses */
>> +#define XEN_HVM_CPUID_X2APIC_VIRT      (1u << 1)
>> +/* Memory mapped from other domains has valid IOMMU entries */
>> +#define XEN_HVM_CPUID_IOMMU_MAPPINGS   (1u << 2)
>> +
>> +#define XEN_CPUID_MAX_NUM_LEAVES 4
>> +
>> +#endif /* __XEN_PUBLIC_ARCH_X86_CPUID_H__ */
>> diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
>> index 1370716..d768371 100644
>> --- a/arch/x86/pci/xen.c
>> +++ b/arch/x86/pci/xen.c
>> @@ -23,6 +23,7 @@
>>   #include <xen/features.h>
>>   #include <xen/events.h>
>>   #include <asm/xen/pci.h>
>> +#include <asm/xen/cpuid.h>
>>   #include <asm/i8259.h>
>>   
>>   static int xen_pcifront_enable_irq(struct pci_dev *dev)
>> @@ -434,6 +435,14 @@ int __init pci_xen_init(void)
>>   #ifdef CONFIG_PCI_MSI
>>   void __init xen_msi_init(void)
>>   {
>> +	if (!disable_apic) {
>> +		uint32_t eax = cpuid_eax(xen_cpuid_base() + 4);
>> +
>> +		if (((eax & XEN_HVM_CPUID_X2APIC_VIRT) && x2apic_mode) ||
>> +		    ((eax & XEN_HVM_CPUID_APIC_ACCESS_VIRT) && cpu_has_apic))
>> +			return;
>> +	}
>> +
>>   	x86_msi.setup_msi_irqs = xen_hvm_setup_msi_irqs;
>>   	x86_msi.teardown_msi_irq = xen_teardown_msi_irq;
>>   }
>> -- 
>> 1.7.1
>>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/