| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20250513203803.2636561-1-sohil.mehta@intel.com>
Date: Tue, 13 May 2025 13:37:54 -0700
From: Sohil Mehta <sohil.mehta@...el.com>
To: x86@...nel.org,
linux-kernel@...r.kernel.org
Cc: Xin Li <xin@...or.com>,
"H . Peter Anvin" <hpa@...or.com>,
Andy Lutomirski <luto@...nel.org>,
Thomas Gleixner <tglx@...utronix.de>,
Ingo Molnar <mingo@...hat.com>,
Borislav Petkov <bp@...en8.de>,
Dave Hansen <dave.hansen@...ux.intel.com>,
Peter Zijlstra <peterz@...radead.org>,
Sean Christopherson <seanjc@...gle.com>,
Adrian Hunter <adrian.hunter@...el.com>,
Kan Liang <kan.liang@...ux.intel.com>,
Tony Luck <tony.luck@...el.com>,
Zhang Rui <rui.zhang@...el.com>,
Steven Rostedt <rostedt@...dmis.org>,
Sohil Mehta <sohil.mehta@...el.com>,
Andrew Cooper <andrew.cooper3@...rix.com>,
"Kirill A . Shutemov" <kirill.shutemov@...ux.intel.com>,
Jacob Pan <jacob.pan@...ux.microsoft.com>,
Andi Kleen <ak@...ux.intel.com>,
Kai Huang <kai.huang@...el.com>,
Sandipan Das <sandipan.das@....com>,
linux-perf-users@...r.kernel.org,
linux-edac@...r.kernel.org,
kvm@...r.kernel.org,
linux-pm@...r.kernel.org,
linux-trace-kernel@...r.kernel.org
Subject: [PATCH v6 0/9] x86: Add support for NMI-source reporting with FRED
Introduction
============
NMI-source reporting with FRED [1] provides a new mechanism for
identifying the source of NMIs. As part of the FRED event delivery
framework, a 16-bit vector bitmap is provided that identifies one or
more sources that caused the NMI.
Using the source bitmap, the kernel can precisely run the relevant NMI
handlers instead of polling the entire NMI handler list. Additionally,
the source information would be invaluable for debugging misbehaving
handlers and unknown NMIs.
Changes since v5
================
This series mainly implements PeterZ's suggestions in patches 4,5,6:
* Simplify NMI handling by always setting and using the source bitmap.
* Add runtime warnings for unexpected values.
* Fix a compile issue in apic.h with a specific config.
* Drop the tracepoint changes for now (include it with the DebugFS series).
* Pick-up Sandipan's tested-by for the perf patch.
The previous posting included a major simplification compared to the
series posted last year[2]. Refer the v5 cover letter for details.
v5: https://lore.kernel.org/lkml/20250507012145.2998143-1-sohil.mehta@intel.com/
Overview of NMI-source usage
============================
Code snippets:
// Allocate a static source vector at compile time
#define NMIS_VECTOR_TEST 1
// Register an NMI handler with the vector
register_nmi_handler(NMI_LOCAL, test_handler, 0, "nmi_test", NMIS_VECTOR_TEST);
// Generate an NMI with the source vector using NMI encoded delivery
__apic_send_IPI_mask(cpumask, APIC_DM_NMI | NMIS_VECTOR_TEST);
// Handle an NMI with or without the source information (oversimplified)
source_bitmap = fred_event_data(regs);
if (!source_bitmap || (source_bitmap & BIT(NMIS_VECTOR_TEST)))
test_handler();
// Unregister handler along with the vector
unregister_nmi_handler(NMI_LOCAL, "nmi_test");
Patch structure
===============
The patches are based on tip:x86/nmi because they depend on the NMI
cleanup series merged earlier [3].
Patch 1-2: Prepare FRED/KVM and enumerate NMI-source reporting
Patch 3-5: Register and handle NMI-source vectors
Patch 6-8: APIC changes to generate NMIs with vectors
Patch 9: Improve debug print with NMI-source information
Many thanks to Sean Christopherson, Xin Li, H. Peter Anvin, Andi Kleen,
Tony Luck, Kan Liang, Jacob Pan Jun, Zeng Guang, Peter Zijlstra,
Sandipan Das, Steven Rostedt and others for their contributions, reviews
and feedback.
Future work / Opens
===================
I am considering a few additional changes that would be valuable for
enhancing NMI handling support. Any feedback, preferences or suggestions
on the following would be helpful.
Assigning more NMI-source vectors
---------------------------------
The current patches assign NMI vectors to a limited number of sources.
The microcode rendezvous and crash reboot code use NMI but do not go
through the typical register_nmi_handler() path. Their handling is
special-cased in exc_nmi(). To isolate blame and improve debugging, it
would be useful to assign vectors to them, even if the vectors are
ignored during handling.
Other NMI sources, such as GHES and Platform NMIs, can also be assigned
vectors to speed up their NMI handling and improve isolation. However,
this would require a software/hardware agreement on vector reservation
and usage. Such an endeavor is likely not worth the effort.
Explicitly enabling NMIs
------------------------
HPA brought up the idea [4] of explicitly enabling NMIs only when the
kernel is ready to take them. With FRED, if we enter the kernel with
NMIs disabled, they could remain disabled until returning back to
userspace. I am evaluating the request and related code changes.
Debug support (Tracing and DebugFS)
-----------------------
NMI-source information can help identify issues when multiple NMIs occur
simultaneously or if certain NMI handlers consistently misbehave. Based
on feedback from Steven Rostedt[5], the plan is to move the trace event
to arch/x86 and then add source_bitmap to the nmi_handler() trace event.
Currently, the kernel has counters for unknown NMIs, swallowed NMIs and
other NMI handling data. However, there is no easy way to access that.
To identify issues that happen over a longer timeframe, it might be
useful to add DebugFS support for NMI statistics.
KVM support
-----------
The NMI-source feature can be useful for perf users and other NMI use
cases in guest VMs. Exposing NMI-source to guests once FRED support is
in place should be relatively easier. The prototype code for this is
under evaluation.
Links
=====
[1]: Chapter 9, https://www.intel.com/content/www/us/en/content-details/819481/flexible-return-and-event-delivery-fred-specification.html
[2]: https://lore.kernel.org/lkml/20240709143906.1040477-1-jacob.jun.pan@linux.intel.com/
[3]: https://lore.kernel.org/lkml/20250327234629.3953536-1-sohil.mehta@intel.com/
[4]: https://lore.kernel.org/lkml/F5D36889-A868-46D2-A678-8EE26E28556D@zytor.com/
[5]: https://lore.kernel.org/lkml/20250507174809.10cfc5ac@gandalf.local.home/
Jacob Pan (1):
perf/x86: Enable NMI-source reporting for perfmon
Sohil Mehta (7):
x86/cpufeatures: Add the CPUID feature bit for NMI-source reporting
x86/nmi: Extend the registration interface to include the NMI-source
vector
x86/nmi: Assign and register NMI-source vectors
x86/nmi: Add support to handle NMIs with source information
x86/nmi: Prepare for the new NMI-source vector encoding
x86/nmi: Enable NMI-source for IPIs delivered as NMIs
x86/nmi: Print source information with the unknown NMI console message
Zeng Guang (1):
x86/fred, KVM: VMX: Pass event data to the FRED entry point from KVM
arch/x86/entry/entry_64_fred.S | 2 +-
arch/x86/events/amd/ibs.c | 2 +-
arch/x86/events/core.c | 6 ++---
arch/x86/events/intel/core.c | 6 ++---
arch/x86/include/asm/apic.h | 39 +++++++++++++++++++++++++++++
arch/x86/include/asm/apicdef.h | 2 +-
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/include/asm/fred.h | 9 ++++---
arch/x86/include/asm/nmi.h | 37 ++++++++++++++++++++++++++-
arch/x86/kernel/apic/hw_nmi.c | 5 ++--
arch/x86/kernel/apic/ipi.c | 4 +--
arch/x86/kernel/apic/local.h | 24 ++++++++++--------
arch/x86/kernel/cpu/cpuid-deps.c | 1 +
arch/x86/kernel/cpu/mce/inject.c | 4 +--
arch/x86/kernel/cpu/mshyperv.c | 3 +--
arch/x86/kernel/kgdb.c | 8 +++---
arch/x86/kernel/kvm.c | 9 +------
arch/x86/kernel/nmi.c | 37 +++++++++++++++++++++++++++
arch/x86/kernel/nmi_selftest.c | 9 +++----
arch/x86/kernel/smp.c | 6 ++---
arch/x86/kvm/vmx/vmx.c | 5 ++--
arch/x86/platform/uv/uv_nmi.c | 4 +--
drivers/acpi/apei/ghes.c | 2 +-
drivers/char/ipmi/ipmi_watchdog.c | 3 +--
drivers/edac/igen6_edac.c | 3 +--
drivers/thermal/intel/therm_throt.c | 2 +-
drivers/watchdog/hpwdt.c | 6 ++---
27 files changed, 171 insertions(+), 68 deletions(-)
base-commit: f2e01dcf6df2d12e86c363ea9c37d53994d89dd6
--
2.43.0
Powered by blists - more mailing lists