Message-ID: <CANpmjNNq1zQ2Lb+mUKHmiOdydE2VjhAVX_GE2G1MjMR_XqgCNQ@mail.gmail.com>
Date: Mon, 8 Dec 2025 10:33:19 +0100
From: Marco Elver <elver@...gle.com>
To: Ard Biesheuvel <ardb@...nel.org>
Cc: Brendan Jackman <jackmanb@...gle.com>, Thomas Gleixner <tglx@...utronix.de>, 
	Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>, 
	Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H. Peter Anvin" <hpa@...or.com>, Nathan Chancellor <nathan@...nel.org>, 
	Nick Desaulniers <nick.desaulniers+lkml@...il.com>, Bill Wendling <morbo@...gle.com>, 
	Justin Stitt <justinstitt@...gle.com>, linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH] x86/sev: Disable GCOV on noinstr object

On Sun, 7 Dec 2025 at 17:41, Ard Biesheuvel <ardb@...nel.org> wrote:
>
> On Sun, 7 Dec 2025 at 03:59, Brendan Jackman <jackmanb@...gle.com> wrote:
> >
> > On Thu Nov 20, 2025 at 3:47 PM UTC, Ard Biesheuvel wrote:
> > > On Thu, 20 Nov 2025 at 12:41, Brendan Jackman <jackmanb@...gle.com> wrote:
> > >>
> > >> On Mon Nov 17, 2025 at 12:37 PM UTC, Brendan Jackman wrote:
> > >> > On Mon, 17 Nov 2025 at 12:52, Ard Biesheuvel <ardb@...nel.org> wrote:
> > >> >>
> > >> >> On Mon, 17 Nov 2025 at 12:40, Ard Biesheuvel <ardb@...nel.org> wrote:
> > >> >> >
> > >> >> > On Mon, 17 Nov 2025 at 12:11, Brendan Jackman <jackmanb@...gle.com> wrote:
> > >> >> > >
> > >> >> > > With Debian clang version 19.1.7 (3+build5) there are calls to
> > >> >> > > kasan_check_write() from __sev_es_nmi_complete, which violates noinstr.
> > >> >> > > Fix it by disabling GCOV for the noinstr object, as has been done for
> > >> >> > > previous such instrumentation issues.
> > >> >> > >
> > >> >> > > Signed-off-by: Brendan Jackman <jackmanb@...gle.com>
> > >> >> > > ---
> > >> >> > > Details:
> > >> >> > >
> > >> >> > >  - ❯❯  clang --version
> > >> >> > >    Debian clang version 19.1.7 (3+build5)
> > >> >> > >    Target: x86_64-pc-linux-gnu
> > >> >> > >    Thread model: posix
> > >> >> > >    InstalledDir: /usr/lib/llvm-19/bin
> > >> >> > >
> > >> >> > >  - Compiling from tip/master at 6f85aad74a70d
> > >> >> > >
> > >> >> > >  - Kernel config:
> > >> >> > >
> > >> >> > >    https://gist.githubusercontent.com/bjackman/bbfdf4ec2e1dfd0e18657174f0537e2c/raw/a88dcc6567d14c69445e7928a7d5dfc23ca9f619/gistfile0.txt
> > >> >> > >
> > >> >> > > Note I also get this error:
> > >> >> > >
> > >> >> > > vmlinux.o: warning: objtool: set_ftrace_ops_ro+0x3b: relocation to !ENDBR: machine_kexec_prepare+0x810
> > >> >> > >
> > >> >> > > That one's a total mystery to me. I guess it's better to "fix" the SEV
> > >> >> > > one independently rather than waiting until I know how to fix them both.
> > >> >> > > ---
> > >> >> > >  arch/x86/coco/sev/Makefile | 3 +++
> > >> >> > >  1 file changed, 3 insertions(+)
> > >> >> > >
> > >> >> > > diff --git a/arch/x86/coco/sev/Makefile b/arch/x86/coco/sev/Makefile
> > >> >> > > index 3b8ae214a6a64de6bb208eb3b7c8bf12007ccc2c..d2ceae587b6c30b2fb17209a7426e7893dea988c 100644
> > >> >> > > --- a/arch/x86/coco/sev/Makefile
> > >> >> > > +++ b/arch/x86/coco/sev/Makefile
> > >> >> > > @@ -8,3 +8,6 @@ UBSAN_SANITIZE_noinstr.o        := n
> > >> >> > >  # GCC may fail to respect __no_sanitize_address or __no_kcsan when inlining
> > >> >> > >  KASAN_SANITIZE_noinstr.o       := n
> > >> >> > >  KCSAN_SANITIZE_noinstr.o       := n
> > >> >> > > +
> > >> >> > > +# Clang 19 and older may fail to respect __no_sanitize_address when inlining
> > >> >> > > +GCOV_PROFILE_noinstr.o         := n
> > >> >> > >
> > >> >> >
> > >> >> > After Thomas dug into this issue a while ago, I meant to follow up
> > >> >> > with a fix, or at least something to start the discussion.
> > >> >> >
> > >> >> > TL;DR there is nothing wrong with either compiler (as far as this
> > >> >> > issue is concerned)
> > >> >> >
> > >> >> > The issue is that KASAN/KCSAN enabled builds use a version of
> > >> >> > set_bit() that unconditionally inserts a call to
> > >> >>
> > >> >> instrument_atomic_write(), which calls the KASAN/KCSAN intrinsics
> > >> >> directly, and these are usually only called by compiler generated
> > >> >> code.
> > >> >>
> > >> >> This completely defeats the noinstr per-function annotation, given
> > >> >> that each compilation unit only incorporates a single version of
> > >> >> set_bit(), which is the instrumented version unless instrumentation is
> > >> >> disabled for the entire file.
> > >> >>
> > >> >> For the short term, we could avoid this by using arch___set_bit()
> > >> >> directly in the SEV code that triggers this issue today. But for the
> > >> >> longer term, we should get write of those explicit calls to
> > >> >> instrumentation intrinsics, as this is fundamentally incompatible with
> > >> >> per-function overrides.
> > >> >>
> > >> >> https://lore.kernel.org/all/8734aqulch.ffs@tglx/T/#u
> > >> >
> > >> > Ah, yes thank you I think you are right. My GCOV "fix" seems to be
> > >> > bogus, it probably just hides the issue with incidental changes.
> > >>
> > >> On the other hand, I guess the intermediate workaround of just disabling
> > >> it at the compilation unit still makes sense here, right?
> > >>
> > >> i.e. my patch is still dumb but should we start by just doing
> > >> K{A,C}ASAN_SANITIZE_noinstr.o := n instead?
> > >
> > > Disabling per file results in the non-instrumented header to be
> > > #include'd, and so we never call the instrument_read/write explicitly.
> > > So yes, this is a reasonable short-term fix.
> >
> > Oh, and we've actually already got K{A,C}SAN_SANITIZE_noinstr.o := n.
> >
> > Ah, I see what's going on here: gcov is injecting instrumentation
> > into kasan_check_write(), which causes it to get out-of-lined even
> > though it's just `return true`:
> >
> >
> >         ❯❯  objdump --disassemble=kasan_check_write ./arch/x86/coco/sev/noinstr.o
> >
> >         ./arch/x86/coco/sev/noinstr.o:     file format elf64-x86-64
> >
> >
> >         Disassembly of section .text:
> >
> >         0000000000000010 <kasan_check_write>:
> >           10:   48 ff 05 00 00 00 00    incq   0x0(%rip)        # 17 <kasan_check_write+0x7>
> >           17:   2e e9 00 00 00 00       cs jmp 1d <kasan_check_write+0xd>
> >
> >         Disassembly of section .noinstr.text:
> >
> >
> > It's the same thing going on with these ones:
> >
> >         vmlinux.o: warning: objtool: do_syscall_64+0x2c3: call to unwind_reset_info() leaves .noinstr.text section
> >         vmlinux.o: warning: objtool: do_int80_emulation+0x311: call to unwind_reset_info() leaves .noinstr.text section
> >         vmlinux.o: warning: objtool: fred_int80_emulation+0x2df: call to unwind_reset_info() leaves .noinstr.text section
> >         vmlinux.o: warning: objtool: __do_fast_syscall_32+0x22a: call to unwind_reset_info() leaves .noinstr.text section
> >         vmlinux.o: warning: objtool: irqentry_exit_to_user_mode+0xc4: call to unwind_reset_info() leaves .noinstr.text section
> >
> >
> >
> > Obvious fix is that we start writing __always_inline even for the stub
> > functions. Seems a bit yucky but I guess it's technically the only
> > correct thing to do?
> >
>
> Either that or a macro. But I think it is always reasonable to replace
> inline with __always_inline if there are cases where the compiler
> ignores it, and this results in build issues.

Why do we need GCOV instrumentation for noinstr.c? Why not just:

GCOV_PROFILE_noinstr.o := n

?

Powered by blists - more mailing lists