| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Tue, 20 May 2014 15:39:59 +0530
From: "Srivatsa S. Bhat" <srivatsa.bhat@...ux.vnet.ibm.com>
To: Peter Zijlstra <peterz@...radead.org>
CC: Oleg Nesterov <oleg@...hat.com>, Tejun Heo <tj@...nel.org>,
akpm@...ux-foundation.org, fweisbec@...il.com,
paulmck@...ux.vnet.ibm.com, tglx@...utronix.de, mingo@...nel.org,
rusty@...tcorp.com.au, hch@...radead.org, mgorman@...e.de,
riel@...hat.com, bp@...e.de, rostedt@...dmis.org,
mgalbraith@...e.de, ego@...ux.vnet.ibm.com, rjw@...ysocki.net,
linux-kernel@...r.kernel.org
Subject: Re: [PATCH v5 UPDATEDv3 3/3] CPU hotplug, smp: Flush any pending
IPI callbacks before CPU offline
On 05/20/2014 03:12 PM, Peter Zijlstra wrote:
> On Tue, May 20, 2014 at 01:52:41AM +0530, Srivatsa S. Bhat wrote:
>> From: Srivatsa S. Bhat <srivatsa.bhat@...ux.vnet.ibm.com>
>> [PATCH v5 UPDATEDv3 3/3] CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline
>>
>> During CPU offline, in the stop-machine loop, we use 2 separate stages to
>> disable interrupts, to ensure that the CPU going offline doesn't get any new
>> IPIs from the other CPUs after it has gone offline.
>>
>> However, an IPI sent much earlier might arrive late on the target CPU
>> (possibly _after_ the CPU has gone offline) due to hardware latencies,
>> and due to this, the smp-call-function callbacks queued on the outgoing
>> CPU might not get noticed (and hence not executed) at all.
>>
>> This is somewhat theoretical, but in any case, it makes sense to explicitly
>> loop through the call_single_queue and flush any pending callbacks before the
>> CPU goes completely offline. So, flush the queued smp-call-function callbacks
>> in the MULTI_STOP_DISABLE_IRQ_ACTIVE stage, after disabling interrupts on the
>> active CPU. This can be trivially achieved by invoking the
>> generic_smp_call_function_single_interrupt() function itself (and since the
>> outgoing CPU is still online at this point, we won't trigger the "IPI to offline
>> CPU" warning in this function; so we are safe to call it here).
>>
>> This way, we would have handled all the queued callbacks before going offline,
>> and also, no new IPIs can be sent by the other CPUs to the outgoing CPU at that
>> point, because they will all be executing the stop-machine code with interrupts
>> disabled.
>>
>> Suggested-by: Frederic Weisbecker <fweisbec@...il.com>
>> Reviewed-by: Tejun Heo <tj@...nel.org>
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@...ux.vnet.ibm.com>
>> ---
>>
>> include/linux/smp.h | 2 ++
>> kernel/smp.c | 17 ++++++++++++++---
>> kernel/stop_machine.c | 11 +++++++++++
>> 3 files changed, 27 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/smp.h b/include/linux/smp.h
>> index 633f5ed..e6b090d 100644
>> --- a/include/linux/smp.h
>> +++ b/include/linux/smp.h
>> @@ -151,6 +151,8 @@ smp_call_function_any(const struct cpumask *mask, smp_call_func_t func,
>>
>> static inline void kick_all_cpus_sync(void) { }
>>
>> +static inline void generic_smp_call_function_single_interrupt(void) { }
>> +
>> #endif /* !SMP */
>>
>> /*
>> diff --git a/kernel/smp.c b/kernel/smp.c
>> index 306f818..b765167 100644
>> --- a/kernel/smp.c
>> +++ b/kernel/smp.c
>> @@ -177,9 +177,18 @@ static int generic_exec_single(int cpu, struct call_single_data *csd,
>> return 0;
>> }
>>
>> -/*
>> - * Invoked by arch to handle an IPI for call function single. Must be
>> - * called from the arch with interrupts disabled.
>> +/**
>> + * generic_smp_call_function_single_interrupt - Execute SMP IPI callbacks
>> + *
>> + * Invoked by arch to handle an IPI for call function single.
>> + *
>> + * This is also invoked by a CPU about to go offline, to flush any pending
>> + * smp-call-function callbacks queued on this CPU (including those for which
>> + * the source CPU's IPIs might not have been received on this CPU yet).
>> + * This ensures that all pending IPI callbacks are run before the CPU goes
>> + * completely offline.
>> + *
>> + * Must be called with interrupts disabled.
>> */
>> void generic_smp_call_function_single_interrupt(void)
>> {
>> @@ -187,6 +196,8 @@ void generic_smp_call_function_single_interrupt(void)
>> struct call_single_data *csd, *csd_next;
>> static bool warned;
>>
>> + WARN_ON(!irqs_disabled());
>> +
>> entry = llist_del_all(&__get_cpu_var(call_single_queue));
>> entry = llist_reverse_order(entry);
>>
>> diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
>> index 288f7fe..4069c13 100644
>> --- a/kernel/stop_machine.c
>> +++ b/kernel/stop_machine.c
>> @@ -21,6 +21,7 @@
>> #include <linux/smpboot.h>
>> #include <linux/atomic.h>
>> #include <linux/lglock.h>
>> +#include <linux/smp.h>
>>
>> /*
>> * Structure to determine completion condition and record errors. May
>> @@ -223,6 +224,16 @@ static int multi_cpu_stop(void *data)
>> if (is_active) {
>> local_irq_disable();
>> hard_irq_disable();
>> +
>> + /*
>> + * IPIs (from the inactive CPUs) might
>> + * arrive late due to hardware latencies.
>> + * So flush out any pending IPI callbacks
>> + * explicitly, to ensure that the outgoing
>> + * CPU doesn't go offline with work still
>> + * pending (during CPU hotplug).
>> + */
>> + generic_smp_call_function_single_interrupt();
>> }
>> break;
>> case MULTI_STOP_RUN:
>
> The multi_cpu_stop() path isn't exclusive to hotplug, so your changelog
> is wrong or the patch is.
>
Yes, I know that multi_cpu_stop() isn't exclusive to hotplug. That's why
I have explicitly referred to CPU hotplug in the comment as well as the
changelog.
But I totally agree that code-wise this is not the best way to do it since
this affects (although harmlessly) usecases other than hotplug as well.
Do you have any other suggestions?
> Furthermore, you yourself have worked on trying to remove stop machine
> from hotplug, that too should've been a big hint this is the wrong place
> for this.
>
I know :-( But for lack of any other solution (other than getting rid of
stop-machine from the hotplug path), I went with this rather poor fix, as
a stop-gap arrangement until we implement the long-term solution (stop-
machine-free CPU hotplug).
While we are on the topic of stop-machine removal, I'll try to revive that
patchset again and hope that this time it will get more reviews and probably
get a chance to go upstream. That should solve a _lot_ of problems with
hotplug!
Regards,
Srivatsa S. Bhat
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists