linux-kernel - Re: [tip:x86/debug] x86, reboot: Use NMI instead of REBOOT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20111221145928.GP5650@redhat.com>
Date:	Wed, 21 Dec 2011 09:59:28 -0500
From:	Don Zickus <dzickus@...hat.com>
To:	Yinghai Lu <yinghai@...nel.org>
Cc:	mingo@...hat.com, hpa@...or.com, linux-kernel@...r.kernel.org,
	andi@...stfloor.org, torvalds@...ux-foundation.org,
	peterz@...radead.org, robert.richter@....com, tglx@...utronix.de,
	mingo@...e.hu, linux-tip-commits@...r.kernel.org
Subject: Re: [tip:x86/debug] x86, reboot: Use NMI instead of REBOOT_VECTOR to
 stop cpus

On Tue, Dec 20, 2011 at 02:38:39PM -0800, Yinghai Lu wrote:
> > @@ -230,7 +285,7 @@ struct smp_ops smp_ops = {
> >        .smp_prepare_cpus       = native_smp_prepare_cpus,
> >        .smp_cpus_done          = native_smp_cpus_done,
> >
> > -       .stop_other_cpus        = native_stop_other_cpus,
> > +       .stop_other_cpus        = native_nmi_stop_other_cpus,
> >        .smp_send_reschedule    = native_smp_send_reschedule,
> >
> >        .cpu_up                 = native_cpu_up,
> 
> this broke kexec on our intel nehalem, westmere and sandbridge platforms.
> system get reset while try to kexec second kernel.


Hmm. Ok.  Does the reboot path work correctly?  Vivek showed me that the
kexec and reboot paths do the same shutdowns.  Perhaps the second kernel
has trouble dealing with cpus spinning in an NMI context and can't
properly reset them.

Let me grab one of those boxes from our lab and try and reproduce the
problem.

Cheers,
Don

> 
> 3603a2512f9e69dc87914ba922eb4a0812b21cd6 is the first bad commit
> commit 3603a2512f9e69dc87914ba922eb4a0812b21cd6
> Author: Don Zickus <dzickus@...hat.com>
> Date:   Thu Oct 13 15:14:25 2011 -0400
> 
>     x86, reboot: Use NMI instead of REBOOT_VECTOR to stop cpus
> 
>     A recent discussion started talking about the locking on the
>     pstore fs and how it relates to the kmsg infrastructure.  We
>     noticed it was possible for userspace to r/w to the pstore fs
>     (grabbing the locks in the process) and block the panic path
>     from r/w to the same fs.
> 
>     The reason was the cpu with the lock could be doing work while
>     the crashing cpu is panic'ing.  Busting those spinlocks might
>     cause those cpus to step on each other's data.  Fine, fair
>     enough.
> 
>     It was suggested it would be nice to serialize the panic path
>     (ie stop the other cpus) and have only one cpu running.  This
>     would allow us to bust the spinlocks and not worry about another
>     cpu stepping on the data.
> 
>     Of course, smp_send_stop() does this in the panic case.
>     kmsg_dump() would have to be moved to be called after it.  Easy
>     enough.
> 
>     The only problem is on x86 the smp_send_stop() function calls
>     the REBOOT_VECTOR.  Any cpu with irqs disabled (which pstore and
>     its backend ERST would do), block this IPI and thus do not stop.
>      This makes it difficult to reliably log data to the pstore fs.
> 
>     The patch below switches from the REBOOT_VECTOR to NMI (and
>     mimics what kdump does).  Switching to NMI allows us to deliver
>     the IPI when irqs are disabled, increasing the reliability of
>     this function.
> 
>     However, Andi carefully noted that on some machines this
>     approach does not work because of broken BIOSes or whatever.
> 
>     To help accomodate this, the next couple of patches will run a
>     selftest and provide a knob to disable.
> 
>     V2:
>       uses atomic ops to serialize the cpu that shuts everyone down
>     V3:
>       comment cleanup
> 
>     Signed-off-by: Don Zickus <dzickus@...hat.com>
>     Cc: Linus Torvalds <torvalds@...ux-foundation.org>
>     Cc: Peter Zijlstra <peterz@...radead.org>
>     Cc: Robert Richter <robert.richter@....com>
>     Cc: seiji.aguchi@....com
>     Cc: vgoyal@...hat.com
>     Cc: mjg@...hat.com
>     Cc: tony.luck@...el.com
>     Cc: gong.chen@...el.com
>     Cc: satoru.moriya@....com
>     Cc: avi@...hat.com
>     Cc: Andi Kleen <andi@...stfloor.org>
>     Link:
> http://lkml.kernel.org/r/1318533267-18880-2-git-send-email-dzickus@redhat.com
>     Signed-off-by: Ingo Molnar <mingo@...e.hu>
> 
> :040000 040000 47a646e9ead83f34fb0728d88c786c875fab91dd
> 6f7a6d5ad8ed686199c0dfbc748ee07a0db97cc5 M      arch
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/