linux-kernel - Re: smp_call_function

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150402095533.GA4968@gmail.com>
Date:	Thu, 2 Apr 2015 11:55:33 +0200
From:	Ingo Molnar <mingo@...nel.org>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	Chris J Arges <chris.j.arges@...onical.com>,
	Rafael David Tinoco <inaddy@...ntu.com>,
	Peter Anvin <hpa@...or.com>,
	Jiang Liu <jiang.liu@...ux.intel.com>,
	Peter Zijlstra <peterz@...radead.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Jens Axboe <axboe@...nel.dk>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Gema Gomez <gema.gomez-solano@...onical.com>,
	the arch/x86 maintainers <x86@...nel.org>
Subject: Re: smp_call_function_single lockups


* Linus Torvalds <torvalds@...ux-foundation.org> wrote:

> On Wed, Apr 1, 2015 at 7:32 AM, Chris J Arges
> <chris.j.arges@...onical.com> wrote:
> >
> > I included the full patch in reply to Ingo's email, and when 
> > running with that I no longer get the ack_APIC_irq WARNs.
> 
> Ok. That means that the printk's themselves just change timing 
> enough, or change the compiler instruction scheduling so that it 
> hides the apic problem.

So another possibility would be that it's the third change causing 
this change in behavior:

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 6cedd7914581..833a981c5420 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -335,9 +340,11 @@ int apic_retrigger_irq(struct irq_data *data)
 
 void apic_ack_edge(struct irq_data *data)
 {
+	ack_APIC_irq();
+
+	/* Might generate IPIs, so do this after having ACKed the APIC: */
 	irq_complete_move(irqd_cfg(data));
 	irq_move_irq(data);
-	ack_APIC_irq();
 }
 
 /*

... since with this we won't send IPIs in a semi-nested fashion with 
an unacked APIC, which is a good idea to do in general. It's also a 
weird enough hardware pattern that virtualization's APIC emulation 
might get it slightly wrong or slightly different.

> Which very much indicates that these things are interconnected.
> 
> For example, Ingo's printk patch does
> 
>                         cfg->move_in_progress =
>                            cpumask_intersects(cfg->old_domain, cpu_online_mask);
> +                       if (cfg->move_in_progress)
> +                               pr_info("apic: vector %02x,
> same-domain move in progress\n", cfg->vector);
>                         cpumask_and(cfg->domain, cfg->domain, tmp_mask);
> 
> and that means that now the setting of move_in_progress is 
> serialized with the cpumask_and() in a way that it wasn't before.

Yeah, that's a possibility too. It all looks very fragile.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/