linux-kernel - Re: [V3 PATCH] x86: avoid calling arch_trigger_all_cpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4CDCA895.7010806@windriver.com>
Date:	Fri, 12 Nov 2010 10:38:13 +0800
From:	DDD <dongdong.deng@...driver.com>
To:	dzickus@...hat.com
CC:	Dongdong Deng <dongdong.deng@...driver.com>, mingo@...e.hu,
	peterz@...radead.org, tglx@...utronix.de, mingo@...hat.com,
	hpa@...or.com, x86@...nel.org, linux-kernel@...r.kernel.org
Subject: Re: [V3 PATCH] x86: avoid calling arch_trigger_all_cpu_backtrace()
 at the same time on SMP

Dongdong Deng wrote:
> The spin_lock_debug/rcu_cpu_stall detector uses
> trigger_all_cpu_backtrace() to dump cpu backtrace.
> Therefore it is possible that trigger_all_cpu_backtrace()
> could be called at the same time on different CPUs, which
> triggers and 'unknown reason NMI' warning. The following case
> illustrates the problem:
> 
>       CPU1                    CPU2                     ...   CPU N
>                        trigger_all_cpu_backtrace()
>                        set "backtrace_mask" to cpu mask
>                                |
> generate NMI interrupts  generate NMI interrupts       ...
>     \                          |                               /
>      \                         |                              /
>    The "backtrace_mask" will be cleaned by the first NMI interrupt
>   at nmi_watchdog_tick(), then the following NMI interrupts generated
> by other cpus's arch_trigger_all_cpu_backtrace() will be took as
> unknown reason NMI interrupts.
> 
> This patch uses a lock to avoid the problem, and stop the
> arch_trigger_all_cpu_backtrace() calling to avoid dumping double cpu
> backtrace info when there is already a trigger_all_cpu_backtrace()
> in progress.
> 
> Signed-off-by: Dongdong Deng <dongdong.deng@...driver.com>
> Reviewed-by: Bruce Ashfield <bruce.ashfield@...driver.com>
> CC: Thomas Gleixner <tglx@...utronix.de>
> CC: Ingo Molnar <mingo@...hat.com>
> CC: "H. Peter Anvin" <hpa@...or.com>
> CC: x86@...nel.org
> CC: linux-kernel@...r.kernel.org
> Signed-off-by: Don Zickus <dzickus@...hat.com>
> ---
>  arch/x86/kernel/apic/hw_nmi.c |   23 +++++++++++++++++++++++
>  arch/x86/kernel/apic/nmi.c    |   23 +++++++++++++++++++++++
>  2 files changed, 46 insertions(+), 0 deletions(-)
> 
> diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
> index cefd694..bfdab3b 100644
> --- a/arch/x86/kernel/apic/hw_nmi.c
> +++ b/arch/x86/kernel/apic/hw_nmi.c
> @@ -26,9 +26,27 @@ u64 hw_nmi_get_sample_period(void)
>  }
>  
>  #ifdef ARCH_HAS_NMI_WATCHDOG
> +/* "in progress" flag of arch_trigger_all_cpu_backtrace */
> +static unsigned long backtrace_flag;
> +
>  void arch_trigger_all_cpu_backtrace(void)
>  {
>  	int i;
> +	unsigned long flags;
> +
> +	/*
> +	 * Have to disable irq here, as the
> +	 * arch_trigger_all_cpu_backtrace() could be
> +	 * triggered by "spin_lock()" with irqs on.
> +	 */
> +	local_irq_save(flags);
> +
> +	if (test_and_set_bit(0, &backtrace_flag))
> +		/*
> +		 * If there is already a trigger_all_cpu_backtrace() in progress
> +		 * (backtrace_flag == 1), don't output double cpu dump infos.
> +		 */
> +		goto out_restore_irq;
>  
>  	cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
>  
> @@ -41,6 +59,11 @@ void arch_trigger_all_cpu_backtrace(void)
>  			break;
>  		mdelay(1);
>  	}
> +
> +	clear_bit(0, &backtrace_flag);

Hi Don,

    The clear_bit() on x86 (arch/x86/include/asm/bitops.h) does not 
contain a memory barrier, Could you add a memory barrier ops here when 
you pick up this patch?

+	clear_bit(0, &backtrace_flag);
+	smp_mb__after_clear_bit();


Thanks,
Dongdong

> +
> +out_restore_irq:
> +	local_irq_restore(flags);
>  }
>  
>  static int __kprobes
> diff --git a/arch/x86/kernel/apic/nmi.c b/arch/x86/kernel/apic/nmi.c
> index c90041c..bd9cb79 100644
> --- a/arch/x86/kernel/apic/nmi.c
> +++ b/arch/x86/kernel/apic/nmi.c
> @@ -549,9 +549,27 @@ int do_nmi_callback(struct pt_regs *regs, int cpu)
>  	return 0;
>  }
>  
> +/* "in progress" flag of arch_trigger_all_cpu_backtrace */
> +static unsigned long backtrace_flag;
> +
>  void arch_trigger_all_cpu_backtrace(void)
>  {
>  	int i;
> +	unsigned long flags;
> +
> +	/*
> +	 * Have to disable irq here, as the
> +	 * arch_trigger_all_cpu_backtrace() could be
> +	 * triggered by "spin_lock()" with irqs on.
> +	 */
> +	local_irq_save(flags);
> +
> +	if (test_and_set_bit(0, &backtrace_flag))
> +		/*
> +		 * If there is already a trigger_all_cpu_backtrace() in progress
> +		 * (backtrace_flag == 1), don't output double cpu dump infos.
> +		 */
> +		goto out_restore_irq;
>  
>  	cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
>  
> @@ -564,4 +582,9 @@ void arch_trigger_all_cpu_backtrace(void)
>  			break;
>  		mdelay(1);
>  	}
> +
> +	clear_bit(0, &backtrace_flag);
> +
> +out_restore_irq:
> +	local_irq_restore(flags);
>  }

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/