Message-ID: <DERNCQGNRITE.139O331ACPKZ9@google.com>
Date: Sun, 07 Dec 2025 02:59:30 +0000
From: Brendan Jackman <jackmanb@...gle.com>
To: Ard Biesheuvel <ardb@...nel.org>, Brendan Jackman <jackmanb@...gle.com>
Cc: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, 
	Borislav Petkov <bp@...en8.de>, Dave Hansen <dave.hansen@...ux.intel.com>, <x86@...nel.org>, 
	"H. Peter Anvin" <hpa@...or.com>, Nathan Chancellor <nathan@...nel.org>, 
	Nick Desaulniers <nick.desaulniers+lkml@...il.com>, Bill Wendling <morbo@...gle.com>, 
	Justin Stitt <justinstitt@...gle.com>, <linux-kernel@...r.kernel.org>, 
	<llvm@...ts.linux.dev>
Subject: Re: [PATCH] x86/sev: Disable GCOV on noinstr object

On Thu Nov 20, 2025 at 3:47 PM UTC, Ard Biesheuvel wrote:
> On Thu, 20 Nov 2025 at 12:41, Brendan Jackman <jackmanb@...gle.com> wrote:
>>
>> On Mon Nov 17, 2025 at 12:37 PM UTC, Brendan Jackman wrote:
>> > On Mon, 17 Nov 2025 at 12:52, Ard Biesheuvel <ardb@...nel.org> wrote:
>> >>
>> >> On Mon, 17 Nov 2025 at 12:40, Ard Biesheuvel <ardb@...nel.org> wrote:
>> >> >
>> >> > On Mon, 17 Nov 2025 at 12:11, Brendan Jackman <jackmanb@...gle.com> wrote:
>> >> > >
>> >> > > With Debian clang version 19.1.7 (3+build5) there are calls to
>> >> > > kasan_check_write() from __sev_es_nmi_complete, which violates noinstr.
>> >> > > Fix it by disabling GCOV for the noinstr object, as has been done for
>> >> > > previous such instrumentation issues.
>> >> > >
>> >> > > Signed-off-by: Brendan Jackman <jackmanb@...gle.com>
>> >> > > ---
>> >> > > Details:
>> >> > >
>> >> > >  - ❯❯  clang --version
>> >> > >    Debian clang version 19.1.7 (3+build5)
>> >> > >    Target: x86_64-pc-linux-gnu
>> >> > >    Thread model: posix
>> >> > >    InstalledDir: /usr/lib/llvm-19/bin
>> >> > >
>> >> > >  - Compiling from tip/master at 6f85aad74a70d
>> >> > >
>> >> > >  - Kernel config:
>> >> > >
>> >> > >    https://gist.githubusercontent.com/bjackman/bbfdf4ec2e1dfd0e18657174f0537e2c/raw/a88dcc6567d14c69445e7928a7d5dfc23ca9f619/gistfile0.txt
>> >> > >
>> >> > > Note I also get this error:
>> >> > >
>> >> > > vmlinux.o: warning: objtool: set_ftrace_ops_ro+0x3b: relocation to !ENDBR: machine_kexec_prepare+0x810
>> >> > >
>> >> > > That one's a total mystery to me. I guess it's better to "fix" the SEV
>> >> > > one independently rather than waiting until I know how to fix them both.
>> >> > > ---
>> >> > >  arch/x86/coco/sev/Makefile | 3 +++
>> >> > >  1 file changed, 3 insertions(+)
>> >> > >
>> >> > > diff --git a/arch/x86/coco/sev/Makefile b/arch/x86/coco/sev/Makefile
>> >> > > index 3b8ae214a6a64de6bb208eb3b7c8bf12007ccc2c..d2ceae587b6c30b2fb17209a7426e7893dea988c 100644
>> >> > > --- a/arch/x86/coco/sev/Makefile
>> >> > > +++ b/arch/x86/coco/sev/Makefile
>> >> > > @@ -8,3 +8,6 @@ UBSAN_SANITIZE_noinstr.o        := n
>> >> > >  # GCC may fail to respect __no_sanitize_address or __no_kcsan when inlining
>> >> > >  KASAN_SANITIZE_noinstr.o       := n
>> >> > >  KCSAN_SANITIZE_noinstr.o       := n
>> >> > > +
>> >> > > +# Clang 19 and older may fail to respect __no_sanitize_address when inlining
>> >> > > +GCOV_PROFILE_noinstr.o         := n
>> >> > >
>> >> >
>> >> > After Thomas dug into this issue a while ago, I meant to follow up
>> >> > with a fix, or at least something to start the discussion.
>> >> >
>> >> > TL;DR there is nothing wrong with either compiler (as far as this
>> >> > issue is concerned)
>> >> >
>> >> > The issue is that KASAN/KCSAN enabled builds use a version of
>> >> > set_bit() that unconditionally inserts a call to
>> >>
>> >> instrument_atomic_write(), which calls the KASAN/KCSAN intrinsics
>> >> directly, and these are usually only called by compiler generated
>> >> code.
>> >>
>> >> This completely defeats the noinstr per-function annotation, given
>> >> that each compilation unit only incorporates a single version of
>> >> set_bit(), which is the instrumented version unless instrumentation is
>> >> disabled for the entire file.
>> >>
>> >> For the short term, we could avoid this by using arch___set_bit()
>> >> directly in the SEV code that triggers this issue today. But for the
>> >> longer term, we should get write of those explicit calls to
>> >> instrumentation intrinsics, as this is fundamentally incompatible with
>> >> per-function overrides.
>> >>
>> >> https://lore.kernel.org/all/8734aqulch.ffs@tglx/T/#u
>> >
>> > Ah, yes thank you I think you are right. My GCOV "fix" seems to be
>> > bogus, it probably just hides the issue with incidental changes.
>>
>> On the other hand, I guess the intermediate workaround of just disabling
>> it at the compilation unit still makes sense here, right?
>>
>> i.e. my patch is still dumb but should we start by just doing
>> K{A,C}ASAN_SANITIZE_noinstr.o := n instead?
>
> Disabling per file results in the non-instrumented header to be
> #include'd, and so we never call the instrument_read/write explicitly.
> So yes, this is a reasonable short-term fix.

Oh, and we've actually already got K{A,C}SAN_SANITIZE_noinstr.o := n.

Ah, I see what's going on here: gcov is injecting instrumentation
into kasan_check_write(), which causes it to get out-of-lined even
though it's just `return true`:


	❯❯  objdump --disassemble=kasan_check_write ./arch/x86/coco/sev/noinstr.o

	./arch/x86/coco/sev/noinstr.o:     file format elf64-x86-64


	Disassembly of section .text:

	0000000000000010 <kasan_check_write>:
	  10:   48 ff 05 00 00 00 00    incq   0x0(%rip)        # 17 <kasan_check_write+0x7>
	  17:   2e e9 00 00 00 00       cs jmp 1d <kasan_check_write+0xd>

	Disassembly of section .noinstr.text:


It's the same thing going on with these ones:

	vmlinux.o: warning: objtool: do_syscall_64+0x2c3: call to unwind_reset_info() leaves .noinstr.text section
	vmlinux.o: warning: objtool: do_int80_emulation+0x311: call to unwind_reset_info() leaves .noinstr.text section
	vmlinux.o: warning: objtool: fred_int80_emulation+0x2df: call to unwind_reset_info() leaves .noinstr.text section
	vmlinux.o: warning: objtool: __do_fast_syscall_32+0x22a: call to unwind_reset_info() leaves .noinstr.text section
	vmlinux.o: warning: objtool: irqentry_exit_to_user_mode+0xc4: call to unwind_reset_info() leaves .noinstr.text section



Obvious fix is that we start writing __always_inline even for the stub
functions. Seems a bit yucky but I guess it's technically the only
correct thing to do?

Maybe there's a way to tell the compiler not to instrument empty
functions? I can't see one though.

Powered by blists - more mailing lists