Message-ID: <CAMj1kXGKe2KsLxaRKmQi75s54y0d9VC1QEFwGfB5zCY8tjjGVA@mail.gmail.com>
Date: Sun, 7 Dec 2025 17:40:59 +0100
From: Ard Biesheuvel <ardb@...nel.org>
To: Brendan Jackman <jackmanb@...gle.com>
Cc: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>, 
	Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H. Peter Anvin" <hpa@...or.com>, Nathan Chancellor <nathan@...nel.org>, 
	Nick Desaulniers <nick.desaulniers+lkml@...il.com>, Bill Wendling <morbo@...gle.com>, 
	Justin Stitt <justinstitt@...gle.com>, linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: [PATCH] x86/sev: Disable GCOV on noinstr object

On Sun, 7 Dec 2025 at 03:59, Brendan Jackman <jackmanb@...gle.com> wrote:
>
> On Thu Nov 20, 2025 at 3:47 PM UTC, Ard Biesheuvel wrote:
> > On Thu, 20 Nov 2025 at 12:41, Brendan Jackman <jackmanb@...gle.com> wrote:
> >>
> >> On Mon Nov 17, 2025 at 12:37 PM UTC, Brendan Jackman wrote:
> >> > On Mon, 17 Nov 2025 at 12:52, Ard Biesheuvel <ardb@...nel.org> wrote:
> >> >>
> >> >> On Mon, 17 Nov 2025 at 12:40, Ard Biesheuvel <ardb@...nel.org> wrote:
> >> >> >
> >> >> > On Mon, 17 Nov 2025 at 12:11, Brendan Jackman <jackmanb@...gle.com> wrote:
> >> >> > >
> >> >> > > With Debian clang version 19.1.7 (3+build5) there are calls to
> >> >> > > kasan_check_write() from __sev_es_nmi_complete, which violates noinstr.
> >> >> > > Fix it by disabling GCOV for the noinstr object, as has been done for
> >> >> > > previous such instrumentation issues.
> >> >> > >
> >> >> > > Signed-off-by: Brendan Jackman <jackmanb@...gle.com>
> >> >> > > ---
> >> >> > > Details:
> >> >> > >
> >> >> > >  - ❯❯  clang --version
> >> >> > >    Debian clang version 19.1.7 (3+build5)
> >> >> > >    Target: x86_64-pc-linux-gnu
> >> >> > >    Thread model: posix
> >> >> > >    InstalledDir: /usr/lib/llvm-19/bin
> >> >> > >
> >> >> > >  - Compiling from tip/master at 6f85aad74a70d
> >> >> > >
> >> >> > >  - Kernel config:
> >> >> > >
> >> >> > >    https://gist.githubusercontent.com/bjackman/bbfdf4ec2e1dfd0e18657174f0537e2c/raw/a88dcc6567d14c69445e7928a7d5dfc23ca9f619/gistfile0.txt
> >> >> > >
> >> >> > > Note I also get this error:
> >> >> > >
> >> >> > > vmlinux.o: warning: objtool: set_ftrace_ops_ro+0x3b: relocation to !ENDBR: machine_kexec_prepare+0x810
> >> >> > >
> >> >> > > That one's a total mystery to me. I guess it's better to "fix" the SEV
> >> >> > > one independently rather than waiting until I know how to fix them both.
> >> >> > > ---
> >> >> > >  arch/x86/coco/sev/Makefile | 3 +++
> >> >> > >  1 file changed, 3 insertions(+)
> >> >> > >
> >> >> > > diff --git a/arch/x86/coco/sev/Makefile b/arch/x86/coco/sev/Makefile
> >> >> > > index 3b8ae214a6a64de6bb208eb3b7c8bf12007ccc2c..d2ceae587b6c30b2fb17209a7426e7893dea988c 100644
> >> >> > > --- a/arch/x86/coco/sev/Makefile
> >> >> > > +++ b/arch/x86/coco/sev/Makefile
> >> >> > > @@ -8,3 +8,6 @@ UBSAN_SANITIZE_noinstr.o        := n
> >> >> > >  # GCC may fail to respect __no_sanitize_address or __no_kcsan when inlining
> >> >> > >  KASAN_SANITIZE_noinstr.o       := n
> >> >> > >  KCSAN_SANITIZE_noinstr.o       := n
> >> >> > > +
> >> >> > > +# Clang 19 and older may fail to respect __no_sanitize_address when inlining
> >> >> > > +GCOV_PROFILE_noinstr.o         := n
> >> >> > >
> >> >> >
> >> >> > After Thomas dug into this issue a while ago, I meant to follow up
> >> >> > with a fix, or at least something to start the discussion.
> >> >> >
> >> >> > TL;DR there is nothing wrong with either compiler (as far as this
> >> >> > issue is concerned)
> >> >> >
> >> >> > The issue is that KASAN/KCSAN enabled builds use a version of
> >> >> > set_bit() that unconditionally inserts a call to
> >> >>
> >> >> instrument_atomic_write(), which calls the KASAN/KCSAN intrinsics
> >> >> directly, and these are usually only called by compiler generated
> >> >> code.
> >> >>
> >> >> This completely defeats the noinstr per-function annotation, given
> >> >> that each compilation unit only incorporates a single version of
> >> >> set_bit(), which is the instrumented version unless instrumentation is
> >> >> disabled for the entire file.
> >> >>
> >> >> For the short term, we could avoid this by using arch___set_bit()
> >> >> directly in the SEV code that triggers this issue today. But for the
> >> >> longer term, we should get write of those explicit calls to
> >> >> instrumentation intrinsics, as this is fundamentally incompatible with
> >> >> per-function overrides.
> >> >>
> >> >> https://lore.kernel.org/all/8734aqulch.ffs@tglx/T/#u
> >> >
> >> > Ah, yes thank you I think you are right. My GCOV "fix" seems to be
> >> > bogus, it probably just hides the issue with incidental changes.
> >>
> >> On the other hand, I guess the intermediate workaround of just disabling
> >> it at the compilation unit still makes sense here, right?
> >>
> >> i.e. my patch is still dumb but should we start by just doing
> >> K{A,C}ASAN_SANITIZE_noinstr.o := n instead?
> >
> > Disabling per file results in the non-instrumented header to be
> > #include'd, and so we never call the instrument_read/write explicitly.
> > So yes, this is a reasonable short-term fix.
>
> Oh, and we've actually already got K{A,C}SAN_SANITIZE_noinstr.o := n.
>
> Ah, I see what's going on here: gcov is injecting instrumentation
> into kasan_check_write(), which causes it to get out-of-lined even
> though it's just `return true`:
>
>
>         ❯❯  objdump --disassemble=kasan_check_write ./arch/x86/coco/sev/noinstr.o
>
>         ./arch/x86/coco/sev/noinstr.o:     file format elf64-x86-64
>
>
>         Disassembly of section .text:
>
>         0000000000000010 <kasan_check_write>:
>           10:   48 ff 05 00 00 00 00    incq   0x0(%rip)        # 17 <kasan_check_write+0x7>
>           17:   2e e9 00 00 00 00       cs jmp 1d <kasan_check_write+0xd>
>
>         Disassembly of section .noinstr.text:
>
>
> It's the same thing going on with these ones:
>
>         vmlinux.o: warning: objtool: do_syscall_64+0x2c3: call to unwind_reset_info() leaves .noinstr.text section
>         vmlinux.o: warning: objtool: do_int80_emulation+0x311: call to unwind_reset_info() leaves .noinstr.text section
>         vmlinux.o: warning: objtool: fred_int80_emulation+0x2df: call to unwind_reset_info() leaves .noinstr.text section
>         vmlinux.o: warning: objtool: __do_fast_syscall_32+0x22a: call to unwind_reset_info() leaves .noinstr.text section
>         vmlinux.o: warning: objtool: irqentry_exit_to_user_mode+0xc4: call to unwind_reset_info() leaves .noinstr.text section
>
>
>
> Obvious fix is that we start writing __always_inline even for the stub
> functions. Seems a bit yucky but I guess it's technically the only
> correct thing to do?
>

Either that or a macro. But I think it is always reasonable to replace
inline with __always_inline if there are cases where the compiler
ignores it, and this results in build issues.

Powered by blists - more mailing lists