linux-kernel - Re: [PATCH RFC 2/2] kvm: set affinity hint for assigned device msi

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20111013145450.GA4143@amt.cnet>
Date:	Thu, 13 Oct 2011 11:54:50 -0300
From:	Marcelo Tosatti <mtosatti@...hat.com>
To:	"Michael S. Tsirkin" <mst@...hat.com>
Cc:	Avi Kivity <avi@...hat.com>, Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>,
	"H. Peter Anvin" <hpa@...or.com>, x86@...nel.org,
	kvm@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH RFC 2/2] kvm: set affinity hint for assigned device msi

On Tue, Oct 11, 2011 at 08:38:28PM +0200, Michael S. Tsirkin wrote:
> To forward an interrupt to a vcpu that runs on
> a host cpu different from the current one,
> we need an ipi which likely will cost us as much
> as delivering the interrupt directly to that cpu would.
> 
> Set irq affinity hint to point there, irq balancer
> can then take this into accound and balance
> interrupts accordingly.
> 
> Signed-off-by: Michael S. Tsirkin <mst@...hat.com>
> ---
>  virt/kvm/assigned-dev.c |    8 +++++---
>  virt/kvm/irq_comm.c     |   17 ++++++++++++++++-
>  2 files changed, 21 insertions(+), 4 deletions(-)
> 
> diff --git a/virt/kvm/assigned-dev.c b/virt/kvm/assigned-dev.c
> index f89f138..b579777 100644
> --- a/virt/kvm/assigned-dev.c
> +++ b/virt/kvm/assigned-dev.c
> @@ -142,9 +142,11 @@ static void deassign_host_irq(struct kvm *kvm,
>  		for (i = 0; i < assigned_dev->entries_nr; i++)
>  			disable_irq(assigned_dev->host_msix_entries[i].vector);
>  
> -		for (i = 0; i < assigned_dev->entries_nr; i++)
> -			free_irq(assigned_dev->host_msix_entries[i].vector,
> -				 (void *)assigned_dev);
> +		for (i = 0; i < assigned_dev->entries_nr; i++) {
> +			u32 vector = assigned_dev->host_msix_entries[i].vector;
> +			irq_set_affinity_hint(vector, NULL);
> +			free_irq(vector, (void *)assigned_dev);
> +		}
>  
>  		assigned_dev->entries_nr = 0;
>  		kfree(assigned_dev->host_msix_entries);
> diff --git a/virt/kvm/irq_comm.c b/virt/kvm/irq_comm.c
> index ac8b629..68b1f7c 100644
> --- a/virt/kvm/irq_comm.c
> +++ b/virt/kvm/irq_comm.c
> @@ -22,6 +22,7 @@
>  
>  #include <linux/kvm_host.h>
>  #include <linux/slab.h>
> +#include <linux/interrupt.h>
>  #include <trace/events/kvm.h>
>  
>  #include <asm/msidef.h>
> @@ -80,6 +81,17 @@ inline static bool kvm_is_dm_lowest_prio(struct kvm_lapic_irq *irq)
>  #endif
>  }
>  
> +static void kvm_vcpu_host_irq_hint(struct kvm_vcpu *vcpu, int host_irq)
> +{
> +	const struct cpumask *mask;
> +	/* raw_smp_processor_id() is ok here: if we get preempted we can get a
> +	 * wrong value but we don't mind much. */
> +	if (host_irq >= 0 && unlikely(vcpu->cpu != raw_smp_processor_id())) {
> +		mask = get_cpu_mask(vcpu->cpu);
> +		irq_set_affinity_hint(host_irq, mask);
> +	}
> +}

Unsure about the internals of irq_set_affinity_hint, but AFAICS its
exported so that irqbalance in userspace can make a decision.

If that is the case, then irqbalance update rate should be high enough
to catch up with a vcpu migrating betweens cpus (which initially does
not appear a sensible arrangement).

The decision to have the host interrupt follow the vcpu seems a good
one, given that it saves an IPI and is potentially more cache friendly
overall.

And AFAICS its more intelligent for the device assignment case than
anything irqbalance can come up with (note it depends on how the APIC is
configured, your patch ignores that).

> +	
>  int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
>  		struct kvm_lapic_irq *irq, int host_irq)
>  {
> @@ -102,6 +114,7 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
>  			if (r < 0)
>  				r = 0;
>  			r += kvm_apic_set_irq(vcpu, irq);
> +			kvm_vcpu_host_irq_hint(vcpu, host_irq);
>  		} else if (kvm_lapic_enabled(vcpu)) {
>  			if (!lowest)
>  				lowest = vcpu;
> @@ -110,8 +123,10 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
>  		}
>  	}
>  
> -	if (lowest)
> +	if (lowest) {
>  		r = kvm_apic_set_irq(lowest, irq);
> +		kvm_vcpu_host_irq_hint(vcpu, host_irq);
> +	}
>  
>  	return r;
>  }
> -- 
> 1.7.5.53.gc233e
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/