| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <4CBFCD03.3080600@windriver.com>
Date: Thu, 21 Oct 2010 13:17:55 +0800
From: DDD <dongdong.deng@...driver.com>
To: Don Zickus <dzickus@...hat.com>
CC: tglx@...utronix.de, hpa@...or.com, mingo@...e.hu, x86@...nel.org,
linux-kernel@...r.kernel.org, bruce.ashfield@...driver.com
Subject: Re: [PATCH] x86: avoid calling arch_trigger_all_cpu_backtrace() at
the same time on SMP
Don Zickus wrote:
> On Mon, Oct 18, 2010 at 07:00:15PM +0800, DDD wrote:
>> CC to Ingo's mingo@...e.hu and add some code explains for this patch.
>>
>> Dongdong
>>
>>
>> Dongdong Deng wrote:
>>> The spin_lock_debug/rcu_cpu_stall detector uses
>>> trigger_all_cpu_backtrace() to dump cpu backtrace.
>>> Therefore it is possible that trigger_all_cpu_backtrace()
>>> could be called at the same time on different CPUs, which
>>> triggers and 'unknown reason NMI' warning. The following case
>>> illustrates the problem:
>
> The patch seems reasonable. I can queue it up. Ingo wrote the original
> code and knows which spin locks to use better than me, perhaps he can find
> a moment to comment.
Hello Don,
Thanks for taking care and queuing this patch. :-)
Dongdong
>
> Cheers,
> Don
>
>>> CPU1 CPU2 ... CPU N
>>> trigger_all_cpu_backtrace()
>>> set "backtrace_mask" to cpu mask
>>> |
>>> generate NMI interrupts generate NMI interrupts ...
>>> \ | /
>>> \ | /
>>> The "backtrace_mask" will be cleaned by the first NMI interrupt
>>> at nmi_watchdog_tick(), then the following NMI interrupts generated
>>> by other cpus's arch_trigger_all_cpu_backtrace() will be took as
>>> unknown reason NMI interrupts.
>>>
>>> This patch uses a lock to avoid the problem, and stop the
>>> arch_trigger_all_cpu_backtrace() calling to avoid dumping double cpu
>>> backtrace info when there is already a trigger_all_cpu_backtrace()
>>> in progress.
>>>
>>> Signed-off-by: Dongdong Deng <dongdong.deng@...driver.com>
>>> Reviewed-by: Bruce Ashfield <bruce.ashfield@...driver.com>
>>> CC: Thomas Gleixner <tglx@...utronix.de>
>>> CC: Ingo Molnar <mingo@...hat.com>
>>> CC: "H. Peter Anvin" <hpa@...or.com>
>>> CC: x86@...nel.org
>>> CC: linux-kernel@...r.kernel.org
>>> ---
>>> arch/x86/kernel/apic/hw_nmi.c | 14 ++++++++++++++
>>> arch/x86/kernel/apic/nmi.c | 14 ++++++++++++++
>>> 2 files changed, 28 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
>>> index cefd694..3aea0a5 100644
>>> --- a/arch/x86/kernel/apic/hw_nmi.c
>>> +++ b/arch/x86/kernel/apic/hw_nmi.c
>>> @@ -29,6 +29,16 @@ u64 hw_nmi_get_sample_period(void)
>>> void arch_trigger_all_cpu_backtrace(void)
>>> {
>>> int i;
>>> + static arch_spinlock_t lock = __ARCH_SPIN_LOCK_UNLOCKED;
>> Why an arch spin lock vs just using a raw spin lock?
>> for example. static DEFINE_RAW_SPINLOCK(lock);
>>
>> The spin_lock_debug detector was used in raw_spinlock too.
>>
>> arch_trigger_all_cpu_backtrace() -->
>> raw_spin_lock(lock) -->
>> _raw_spin_lock(lock) -->
>> __raw_spin_lock(lock) -->
>> do_raw_spin_lock(lock) -->
>> __spin_lock_debug(lock) -->
>> trigger_all_cpu_backtrace()
>>
>> Therefor, we have to use arch spin lock here.
>>
>>
>>
>>> + unsigned long flags;
>>> +
>>> + local_irq_save(flags);
>> Why have to save the irq's here?
>>
>> When the arch_trigger_all_cpu_backtrace() was triggered by
>> "spin_lock()"'s spin_lock_debug detector, it is possible that
>> the irq is enabled, thus we have to save and disable it here.
>>
>>
>>
>>> + if (!arch_spin_trylock(&lock))
>>> + /*
>>> + * If there is already a trigger_all_cpu_backtrace()
>>> + * in progress, don't output double cpu dump infos.
>>> + */
>>> + goto out_restore_irq;
>>> cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
>>> @@ -41,6 +51,10 @@ void arch_trigger_all_cpu_backtrace(void)
>>> break;
>>> mdelay(1);
>>> }
>>> +
>>> + arch_spin_unlock(&lock);
>>> +out_restore_irq:
>>> + local_irq_restore(flags);
>>> }
>>> static int __kprobes
>>> diff --git a/arch/x86/kernel/apic/nmi.c b/arch/x86/kernel/apic/nmi.c
>>> index a43f71c..5fa8a13 100644
>>> --- a/arch/x86/kernel/apic/nmi.c
>>> +++ b/arch/x86/kernel/apic/nmi.c
>>> @@ -552,6 +552,16 @@ int do_nmi_callback(struct pt_regs *regs, int cpu)
>>> void arch_trigger_all_cpu_backtrace(void)
>>> {
>>> int i;
>>> + static arch_spinlock_t lock = __ARCH_SPIN_LOCK_UNLOCKED;
>>> + unsigned long flags;
>>> +
>>> + local_irq_save(flags);
>>> + if (!arch_spin_trylock(&lock))
>>> + /*
>>> + * If there is already a trigger_all_cpu_backtrace()
>>> + * in progress, don't output double cpu dump infos.
>>> + */
>>> + goto out_restore_irq;
>>> cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
>>> @@ -564,4 +574,8 @@ void arch_trigger_all_cpu_backtrace(void)
>>> break;
>>> mdelay(1);
>>> }
>>> +
>>> + arch_spin_unlock(&lock);
>>> +out_restore_irq:
>>> + local_irq_restore(flags);
>>> }
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@...r.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists