linux-kernel - CPU hotplug and chained interrupts on x86

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151001142147.GO1551@lahna.fi.intel.com>
Date:	Thu, 1 Oct 2015 17:21:47 +0300
From:	Mika Westerberg <mika.westerberg@...ux.intel.com>
To:	Thomas Gleixner <tglx@...utronix.de>
Cc:	Jiang Liu <jiang.liu@...ux.intel.com>, linux-kernel@...r.kernel.org
Subject: CPU hotplug and chained interrupts on x86

Hi Thomas,

On Intel Braswell system (this affect others if they are using chained
interrupts on x86) both CPUs can handle interrupts which trigger when
GPIO line changes state if programmed to do so.

We have SD-card card detection signal that is connected to a GPIO line:

  # cat /proc/interrupts 
              CPU0       CPU1       
   ...
   304:          0          0  chv-gpio   50  80860F14:01 cd

This works fine until the other CPU is offlined.

  # echo 0 > /sys/devices/system/cpu/cpu1/online

I modified arch/x86/kernel/irq.c:fixup_irqs() slightly so that it calls
print_IO_APICs() at the end to be able to see how interrupts are routed
after the CPU is offlined (below lists entries related to the four
interrupts used by the GPIO controller):

  IOAPIC 0:
   pin30, enabled , level, low , V(52), IRR(0), S(0), logical , D(03), M(1)
   pin31, enabled , level, low , V(42), IRR(0), S(0), logical , D(03), M(1)
   pin32, enabled , level, low , V(62), IRR(0), S(0), logical , D(03), M(1)
   pin5b, enabled , level, low , V(72), IRR(0), S(0), logical , D(03), M(1)

The destination mask D(03) says that the interrupt can be delivered to any
of the two CPUs and mode M(1) says to deliver it to lowest priority CPU
among the list.

Now if I plug/unplug the card I may get few interrupts to CPU0 but rest
of the interrupts never happen. Probably because IO-APIC forwards them
to the lowest priority CPU which is offline at this point.

There is following check in fixup_irqs():

	if (!irq_has_action(irq) || irqd_is_per_cpu(data) ||
	    cpumask_subset(affinity, cpu_online_mask)) {
		raw_spin_unlock(&desc->lock);
		continue;
	}   

If an interrupt is requested by a driver it will force new affinity and
everything works fine. However if the interrupt is chained (it does not
have ->action) this is skipped and the current affinity remains.

I'm able to work this around by forcing the affinity here for the 4 chained
interrupts. However, I'm not quite sure what would the proper fix be.

We could detect here if the interrupt is chained but there seems to be
no easy way to determine it currently so we would need to add a new flag
to desc->status_use_accessors that gets set in __irq_do_set_handler()
when is_chained is 1.

Alternative I could implement ->irq_set_affinity() in the GPIO driver in
question [1] which always calls directly parent chip's ->irq_set_affinity()
but I'm not sure if that is allowed.

Any ideas how to get this properly fixed?

Thanks in advance.

[1] drivers/pinctrl/intel/pinctrl-cherryview.c
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/