linux-kernel - Re: [PATCH v2 0/4] KVM: APIC improvements (with bonus mixed mode)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150226015526.GA26845@amt.cnet>
Date:	Wed, 25 Feb 2015 22:55:26 -0300
From:	Marcelo Tosatti <mtosatti@...hat.com>
To:	Radim Krčmář <rkrcmar@...hat.com>
Cc:	linux-kernel@...r.kernel.org, kvm@...r.kernel.org,
	Paolo Bonzini <pbonzini@...hat.com>,
	Nadav Amit <namit@...technion.ac.il>,
	Gleb Natapov <gleb@...nel.org>
Subject: Re: [PATCH v2 0/4] KVM: APIC improvements (with bonus mixed mode)

Radim,

On Thu, Feb 12, 2015 at 07:41:30PM +0100, Radim Krčmář wrote:
> Each patch has a diff from v1, here is only a prologue on the mythical
> mixed xAPIC and x2APIC mode:
> 
> There is one interesting alias in xAPIC and x2APIC ICR destination, the
> 0xff000000, which is a broadcast in xAPIC and either a physical message
> to high x2APIC ID or a message to an empty set in x2APIC cluster.
> 
> This corner case in mixed mode can be triggered by a weird message
>  1) when we have no x2APIC ID 0xff000000, but send x2APIC message there

10.7 SYSTEM AND APIC BUS ARBITRATION

Note that except for the SIPI IPI (see Section 10.6.1, “Interrupt
Command Register (ICR)”), all bus messages that fail to be delivered
to their specified destination or destinations are automatically
retried. Software should avoid situations in which IPIs are sent to
disabled or nonexistent local APICs, causing the messages to be resent
repeatedly.

> or after something that isn't forbidden in SDM most likely because they
> didn't think that anyone would ever consider it
>  2) we have x2APIC ID 0xff000000 and reset other x2APICs into xAPIC mode

Or just x2APIC initialization in non lockstep:

	vcpu0				vcpu1
T0) 	xapic mode			xapic mode
T1)	x2apic enable
T2)					broadcast IPI

> Current KVM doesn't need to consider (2), so there only is a slim chance
> that some hobbyist OS pushes the specification to its limits.
> 
> The problem is that SDM for xAPIC doesn't believably specify[1] if all
> messages beginning with 0xff are considered as broadcasts, 10.6.2.1:
>   A broadcast IPI (bits 28-31 of the MDA are 1's)
> 
> No volunteer came to clear this up, so I hacked Linux to have one x2APIC
> core between xAPIC ones.  Physical IPI to 0xff000000 got delivered only
> to CPU0, like most other destinations, Logical IPI to 0xff000000 got
> dropped and only 0xffffffff worked as a broadcast in both modes.
> 
> I think it is because Intel never considered mixed mode to be valid, and
> seen delivery might be an emergent feature of QPI routing.

> Luckily, broadcast from xAPIC isn't delivered to x2APIC.

In real hardware?

> Real exploration would require greater modifications to Linux (ideally
> writing a custom kernel), so this series implements something that makes
> some sense and isn't too far from reality.

Ok, so the problem is that broadcast (ICR.destination == 0xff000000)
from xAPIC CPU is not delivered to x2APIC CPUs ?

> Radim Krčmář (4):
>   KVM: x86: use MDA for interrupt matching
>   KVM: x86: fix mixed APIC mode broadcast
>   KVM: x86: avoid logical_map when it is invalid
>   KVM: x86: simplify kvm_apic_map

I can't find any restriction against (cpu0==x2APIC, cpu1==xAPIC) in 
the documentation.

Anyway, emulation should match physical hardware. From your message above,
it is not clear what is the behaviour there:

"No volunteer came to clear this up, so I hacked Linux to have one x2APIC
core between xAPIC ones.  Physical IPI to 0xff000000 got delivered only
to CPU0, like most other destinations, Logical IPI to 0xff000000 got
dropped and only 0xffffffff worked as a broadcast in both modes.

Luckily, broadcast from xAPIC isn't delivered to x2APIC."

>From the x2APIC CPU or the xAPIC ones?

It should be easy to write kvm-unit-test testcases that match physical
hardware behaviour (in general, i am having a hard time figure out
in what way "mixed mode" is supposed to behave, please describe it
clearly).

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/