Message-ID: <m14pf7i4ik.fsf@ebiederm.dsl.xmission.com>
Date:	Tue, 27 Nov 2007 17:43:15 -0700
From:	ebiederm@...ssion.com (Eric W. Biederman)
To:	Neil Horman <nhorman@...hat.com>
Cc:	Ben Woodard <woodard@...hat.com>,
	Neil Horman <nhorman@...driver.com>, kexec@...ts.infradead.org,
	Andi Kleen <ak@...e.de>, linux-kernel@...r.kernel.org,
	hbabu@...ibm.com, vgoyal@...ibm.com
Subject: Re: [PATCH] kexec: force x86_64 arches to boot kdump kernels on boot cpu

Neil Horman <nhorman@...hat.com> writes:

>> Close.  There are two options with virtual wire mode.  
>> - Either the local apic is in virtual wire mode, and somehow the
>>   legacy interrupts make it to the local cpu.

> I assume this is the case if the ioapic is also in virtual wire
> mode.

No.  The ioapic is completely disabled in this case.

> and the
> destination field for the appropriate interrupt(s) (the timer interrupt in this
> case) is set to either physical mode with a destination id of the lapic for the
> running cpu, or if it is set to logical mode and the destination id has the
> corresponding bit for the running cpu set.  Is that right?

No.  All of the ioapic routing entries are disabled.

>> - Or an ioapic is in virtual wire mode and the legacy interrupt
>>   controller is connected to it.
>> 
> I thought we only had one ioapic in this system (Ben correct me if I'm wrong on
> that please).  I thought the above printk told us that, because apic2 and pin2
> are both -1, that means that the 8259 isn't physically connected to any cpu, and
> instead is routed through apic 0, and asserts on pin 2 of that ioapic. 
>
>> So I guess fundamentally for any SMP system that only supports the
>> cpu being in local apic mode and only routes interrupts to the boot
>> strap processor we could be in trouble.  That is what our current
>> information about your system suggests.
>> 
> If that were the case, then we would need to support moving kexec boot to cpu0,
> at least in some limited cases.  I've got a patch together that enables the
> handshaking I was brainstorming earlier, which should allow an attempted jump to
> cpu0 on a crash, with a fallback to booting on the crashing processor.  If we
> wind up confirming the above case, then I'll post it.
>
>> However most systems actually connect the i8254 PIC interrupt
> Sorry, to split hairs here, but you mean the 8259 right? 

Grr. Yes. The i8259.  Got the timer and the PIC numbers confused Oops.
The legacy configuration is the timer to the PIC to either the local
or the ioapic.

> Just want to be sure I'm clear on whats going on.  I thought the
> 8254 was the external timer. 

Yep.  You have it straight.

>> controller to the ioapic in virtual wire mode.  As I recall the
>> standard mapping is to ioapic 0, pin 0.  With ioapic 0, pin 2 being
>> the timer interrupt (Possibly it is the other way around).
>> 
>> So as a test we could feed those values into ioapic_8259 and see
>> if the kdump case works.  I believe we prefer putting the ioapic
>> into virtual wire mode over putting the cpu into virtual wire
>> mode.  We can only control which cpu receives the legacy interrupts if
>> we are putting the ioapic in virtual wire mode.
>> 
> I'm sorry, I can't find ioapic_8259 defined anywhere.  Where is that supposed to
> be?  Show me where its defined and I'll happily write the patch.

Grr.  That should have been ioapic_i8259.  The second value we print out
in the ..Timer printk.

>> It may also be an interesting test to just enable the timer for the
>> ioapic in early boot, as you have suggested.  I don't have a clue what
>> that will do.
>> 
> Unfortunately nothing.  We've tried using the local apic timer in a previous
> test, and it resulted in no change, as did transitioning the cpu to the apic
> timer via a call to switch_ipi_to_APIC_timer.  Its possible I did something
> wrong however.

Well if you didn't have the local apic enabled beyond virtual wire
mode that could have cause problems.

Sure.  I suspect the probably is that you were still in a legacy irq
mode.

> Currently I'm writing a patch that calls setup_ioapic_dest after we call
> disable_IO_APIC.  Looking at the implementation, it appears that calling this
> function should rewrite the irq routing table in the ioapic to deliver
> interrupts to the set of online cpus, as defined by the TARGET_CPUS macro.  I
> asusme that if the ioapic is in virtual wire mode from the call to
> disablie_IO_APIC, then calling setup_ioapic_dest will force interrupts to be
> delivered to the crashing cpu, as it should be the only bit set in the online
> cpu mask.  Please feel free to poke holes in this idea.

Just try and make certain: ioapic_i8259.pin != -1
Which should cause disable_IO_APIC to put the ioapic and not the local
apic in virtual wire mode.

Anything else is likely to do strange things.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists