| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <1603983387-8738-1-git-send-email-sumit.garg@linaro.org>
Date: Thu, 29 Oct 2020 20:26:20 +0530
From: Sumit Garg <sumit.garg@...aro.org>
To: maz@...nel.org, catalin.marinas@....com, will@...nel.org
Cc: linux-arm-kernel@...ts.infradead.org, tglx@...utronix.de,
jason@...edaemon.net, linux@...linux.org.uk,
tsbogend@...ha.franken.de, mpe@...erman.id.au, davem@...emloft.net,
mingo@...hat.com, bp@...en8.de, x86@...nel.org,
mark.rutland@....com, julien.thierry.kdev@...il.com,
dianders@...omium.org, daniel.thompson@...aro.org,
jason.wessel@...driver.com, msys.mizuma@...il.com,
ito-yuichi@...itsu.com, kgdb-bugreport@...ts.sourceforge.net,
linux-kernel@...r.kernel.org, Sumit Garg <sumit.garg@...aro.org>
Subject: [PATCH v6 0/7] arm64: Add framework to turn an IPI as NMI
With pseudo NMIs support available its possible to configure SGIs to be
triggered as pseudo NMIs running in NMI context. And kernel features
such as:
- NMI backtrace can leverage IPI turned as NMI to get a backtrace of CPU
stuck in hard lockup using magic SYSRQ.
- kgdb relies on NMI support to round up CPUs which are stuck in hard
lockup state with interrupts disabled.
This patch-set adds framework to turn an IPI as NMI which can be triggered
as a pseudo NMI which in turn invokes registered NMI handlers.
After this patch-set we should be able to get a backtrace for a CPU
stuck in HARDLOCKUP. Have a look at an examples below from a hard lockup
testcase run on Developerbox:
$ echo HARDLOCKUP > /sys/kernel/debug/provoke-crash/DIRECT
NMI backtrace:
==============
# Issue Magic SysRq to dump backtrace
[ 376.894502] NMI backtrace for cpu 8
[ 376.894506] CPU: 8 PID: 555 Comm: bash Not tainted 5.9.0-rc3-00740-g06ff047-dirty #242
[ 376.894510] Hardware name: Socionext SynQuacer E-series DeveloperBox, BIOS build #73 Apr 6 2020
[ 376.894514] pstate: 40000005 (nZcv daif -PAN -UAO BTYPE=--)
[ 376.894517] pc : lkdtm_HARDLOCKUP+0x8/0x18
[ 376.894520] lr : lkdtm_do_action+0x24/0x30
[ 376.894524] sp : ffff800012cebd20
[ 376.894527] pmr_save: 00000060
[ 376.894530] x29: ffff800012cebd20 x28: ffff000875ae8000
[ 376.894540] x27: 0000000000000000 x26: 0000000000000000
[ 376.894550] x25: 000000000000001a x24: ffff800012cebe40
[ 376.894560] x23: 000000000000000b x22: ffff800010fc5040
[ 376.894569] x21: ffff000878b61000 x20: ffff8000113b2870
[ 376.894579] x19: 000000000000001b x18: 0000000000000010
[ 376.894588] x17: 0000000000000000 x16: 0000000000000000
[ 376.894598] x15: ffff000875ae8470 x14: 00000000000002ad
[ 376.894613] x13: 0000000000000000 x12: 0000000000000000
[ 376.894622] x11: 0000000000000007 x10: 00000000000009c0
[ 376.894631] x9 : ffff800012ceba80 x8 : ffff000875ae8a20
[ 376.894641] x7 : ffff00087f6b3280 x6 : ffff00087f6b3200
[ 376.894651] x5 : 0000000000000000 x4 : ffff00087f6a91f8
[ 376.894660] x3 : ffff00087f6b0120 x2 : 1aa310cec69eb500
[ 376.894670] x1 : 0000000000000000 x0 : 0000000000000060
[ 376.894679] Call trace:
[ 376.894683] lkdtm_HARDLOCKUP+0x8/0x18
[ 376.894686] direct_entry+0x124/0x1c0
[ 376.894689] full_proxy_write+0x60/0xb0
[ 376.894693] vfs_write+0xf0/0x230
[ 376.894696] ksys_write+0x6c/0xf8
[ 376.894699] __arm64_sys_write+0x1c/0x28
[ 376.894703] el0_svc_common.constprop.0+0x74/0x1f0
[ 376.894707] do_el0_svc+0x24/0x90
[ 376.894710] el0_sync_handler+0x180/0x2f8
[ 376.894713] el0_sync+0x158/0x180
KGDB:
=====
# Enter kdb via Magic SysRq
[6]kdb> btc
btc: cpu status: Currently on cpu 6
Available cpus: 0-5(I), 6, 7(I), 8, 9-23(I)
<snip>
Stack traceback for pid 555
0xffff000875ae8000 555 554 1 8 R 0xffff000875ae89c0 bash
CPU: 8 PID: 555 Comm: bash Not tainted 5.9.0-rc3-00740-g06ff047-dirty #242
Hardware name: Socionext SynQuacer E-series DeveloperBox, BIOS build #73 Apr 6 2020
Call trace:
dump_backtrace+0x0/0x1a0
show_stack+0x18/0x28
dump_stack+0xc0/0x11c
kgdb_cpu_enter+0x648/0x660
kgdb_nmicallback+0xa0/0xa8
ipi_kgdb_nmicallback+0x24/0x30
ipi_nmi_handler+0x48/0x60
handle_percpu_devid_fasteoi_ipi+0x74/0x88
generic_handle_irq+0x30/0x48
handle_domain_nmi+0x48/0x80
gic_handle_irq+0x18c/0x34c
el1_irq+0xcc/0x180
lkdtm_HARDLOCKUP+0x8/0x18
direct_entry+0x124/0x1c0
full_proxy_write+0x60/0xb0
vfs_write+0xf0/0x230
ksys_write+0x6c/0xf8
__arm64_sys_write+0x1c/0x28
el0_svc_common.constprop.0+0x74/0x1f0
do_el0_svc+0x24/0x90
el0_sync_handler+0x180/0x2f8
el0_sync+0x158/0x180
<snip>
Changes in v6:
- Two new patches: #4 and #6 which adds runtime fallback framework for
sysrq backtrace and kgdb roundup features.
- Reversed order of NMI backtrace and kgdb roundup feaure patches.
- Addressed other misc. comments from Marc.
- I haven't picked any tags from v5 since I think there is major rework
involved. Masayoshi, could you please confirm if these features still
work for you?
Changes in v5:
- Rebased to head of upstream master.
- Remove redundant invocation of ipi_nmi_setup().
- Addressed misc. comments.
Changes in v4:
- Move IPI NMI framework to a separate file.
- Get rid of hard-coded IPI_CALL_NMI_FUNC allocation.
- Add NMI backtrace support leveraged via magic SYSRQ.
Changes in v3:
- Rebased to Marc's latest IPIs patch-set [1].
[1] https://lkml.org/lkml/2020/9/1/603
Changes since RFC version [1]:
- Switch to use generic interrupt framework to turn an IPI as NMI.
- Dependent on Marc's patch-set [2] which turns IPIs into normal
interrupts.
- Addressed misc. comments from Doug on patch #4.
- Posted kgdb NMI printk() fixup separately which has evolved since
to be solved using different approach via changing kgdb interception
of printk() in common printk() code (see patch [3]).
[1] https://lkml.org/lkml/2020/4/24/328
[2] https://lkml.org/lkml/2020/5/19/710
[3] https://lkml.org/lkml/2020/5/20/418
Sumit Garg (7):
arm64: Add framework to turn IPI as NMI
irqchip/gic-v3: Enable support for SGIs to act as NMIs
arm64: smp: Assign and setup an IPI as NMI
nmi: backtrace: Allow runtime arch specific override
arm64: ipi_nmi: Add support for NMI backtrace
kgdb: roundup: Allow runtime arch specific override
arm64: kgdb: Roundup cpus using IPI as NMI
arch/arm/include/asm/irq.h | 2 +-
arch/arm/kernel/smp.c | 3 +-
arch/arm64/include/asm/irq.h | 6 +++
arch/arm64/include/asm/kgdb.h | 9 +++++
arch/arm64/include/asm/nmi.h | 17 ++++++++
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/ipi_nmi.c | 84 ++++++++++++++++++++++++++++++++++++++++
arch/arm64/kernel/kgdb.c | 35 +++++++++++++++++
arch/arm64/kernel/smp.c | 8 ++++
arch/mips/include/asm/irq.h | 2 +-
arch/mips/kernel/process.c | 3 +-
arch/powerpc/include/asm/nmi.h | 2 +-
arch/powerpc/kernel/kgdb.c | 3 +-
arch/powerpc/kernel/stacktrace.c | 3 +-
arch/sparc/include/asm/irq_64.h | 2 +-
arch/sparc/kernel/process_64.c | 4 +-
arch/sparc/kernel/smp_64.c | 3 +-
arch/x86/include/asm/irq.h | 2 +-
arch/x86/kernel/apic/hw_nmi.c | 3 +-
arch/x86/kernel/kgdb.c | 6 ++-
drivers/irqchip/irq-gic-v3.c | 29 ++++++++++----
include/linux/kgdb.h | 5 ++-
include/linux/nmi.h | 12 ++----
kernel/debug/debug_core.c | 10 ++++-
24 files changed, 221 insertions(+), 34 deletions(-)
create mode 100644 arch/arm64/include/asm/nmi.h
create mode 100644 arch/arm64/kernel/ipi_nmi.c
--
2.7.4
Powered by blists - more mailing lists