linux-kernel - Re: [PATCH v2 0/4] KVM: APIC improvements (with bonus mixed mode)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <0F7F03BA-B071-445B-AE33-4F9D95363CF9@gmail.com>
Date:	Thu, 26 Feb 2015 10:49:53 +0200
From:	Nadav Amit <nadav.amit@...il.com>
To:	Marcelo Tosatti <mtosatti@...hat.com>
Cc:	Radim Krčmář <rkrcmar@...hat.com>,
	linux-kernel@...r.kernel.org, kvm@...r.kernel.org,
	Paolo Bonzini <pbonzini@...hat.com>,
	Nadav Amit <namit@...technion.ac.il>,
	Gleb Natapov <gleb@...nel.org>
Subject: Re: [PATCH v2 0/4] KVM: APIC improvements (with bonus mixed mode)

Marcello, Radim,

As you know - I can run some tests on the patches and whether they comply
with real hardware. Please let me know which version to test and I’ll try to
do next week.

Regards,
Nadav

Marcelo Tosatti <mtosatti@...hat.com> wrote:

> Radim,
> 
> On Thu, Feb 12, 2015 at 07:41:30PM +0100, Radim Krčmář wrote:
>> Each patch has a diff from v1, here is only a prologue on the mythical
>> mixed xAPIC and x2APIC mode:
>> 
>> There is one interesting alias in xAPIC and x2APIC ICR destination, the
>> 0xff000000, which is a broadcast in xAPIC and either a physical message
>> to high x2APIC ID or a message to an empty set in x2APIC cluster.
>> 
>> This corner case in mixed mode can be triggered by a weird message
>> 1) when we have no x2APIC ID 0xff000000, but send x2APIC message there
> 
> 10.7 SYSTEM AND APIC BUS ARBITRATION
> 
> Note that except for the SIPI IPI (see Section 10.6.1, “Interrupt
> Command Register (ICR)”), all bus messages that fail to be delivered
> to their specified destination or destinations are automatically
> retried. Software should avoid situations in which IPIs are sent to
> disabled or nonexistent local APICs, causing the messages to be resent
> repeatedly.
> 
>> or after something that isn't forbidden in SDM most likely because they
>> didn't think that anyone would ever consider it
>> 2) we have x2APIC ID 0xff000000 and reset other x2APICs into xAPIC mode
> 
> Or just x2APIC initialization in non lockstep:
> 
> 
> 	vcpu0				vcpu1
> T0) 	xapic mode			xapic mode
> T1)	x2apic enable
> T2)					broadcast IPI
> 
>> Current KVM doesn't need to consider (2), so there only is a slim chance
>> that some hobbyist OS pushes the specification to its limits.
>> 
>> The problem is that SDM for xAPIC doesn't believably specify[1] if all
>> messages beginning with 0xff are considered as broadcasts, 10.6.2.1:
>>  A broadcast IPI (bits 28-31 of the MDA are 1's)
>> 
>> No volunteer came to clear this up, so I hacked Linux to have one x2APIC
>> core between xAPIC ones.  Physical IPI to 0xff000000 got delivered only
>> to CPU0, like most other destinations, Logical IPI to 0xff000000 got
>> dropped and only 0xffffffff worked as a broadcast in both modes.
>> 
>> I think it is because Intel never considered mixed mode to be valid, and
>> seen delivery might be an emergent feature of QPI routing.
> 
>> Luckily, broadcast from xAPIC isn't delivered to x2APIC.
> 
> In real hardware?
> 
>> Real exploration would require greater modifications to Linux (ideally
>> writing a custom kernel), so this series implements something that makes
>> some sense and isn't too far from reality.
> 
> Ok, so the problem is that broadcast (ICR.destination == 0xff000000)
> from xAPIC CPU is not delivered to x2APIC CPUs ?
> 
>> Radim Krčmář (4):
>>  KVM: x86: use MDA for interrupt matching
>>  KVM: x86: fix mixed APIC mode broadcast
>>  KVM: x86: avoid logical_map when it is invalid
>>  KVM: x86: simplify kvm_apic_map
> 
> I can't find any restriction against (cpu0==x2APIC, cpu1==xAPIC) in 
> the documentation.
> 
> Anyway, emulation should match physical hardware. From your message above,
> it is not clear what is the behaviour there:
> 
> "No volunteer came to clear this up, so I hacked Linux to have one x2APIC
> core between xAPIC ones.  Physical IPI to 0xff000000 got delivered only
> to CPU0, like most other destinations, Logical IPI to 0xff000000 got
> dropped and only 0xffffffff worked as a broadcast in both modes.
> 
> Luckily, broadcast from xAPIC isn't delivered to x2APIC."
> 
> From the x2APIC CPU or the xAPIC ones?
> 
> It should be easy to write kvm-unit-test testcases that match physical
> hardware behaviour (in general, i am having a hard time figure out
> in what way "mixed mode" is supposed to behave, please describe it
> clearly).
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/