Date:   Tue, 07 Feb 2023 21:38:44 +0100
From:   "Arnd Bergmann" <arnd@...db.de>
To:     "Peter Zijlstra" <peterz@...radead.org>
Cc:     "Josh Poimboeuf" <jpoimboe@...nel.org>,
        linux-kernel@...r.kernel.org, llvm@...ts.linux.dev
Subject: Re: current objtool warnings from randconfig builds

On Tue, Feb 7, 2023, at 17:16, Peter Zijlstra wrote:
>
>> 80 vmlinux.o: warning: objtool: ibt_selftest+0x11: sibling call from callable instruction with modified stack frame
>
> Can you provide details on how to reproduce this?

I see this one with clang-14 and clang-16, presumably other versions
but not gcc, the function disassembles the clang version into

0000000000000140 <ibt_selftest>:
 140:   f3 0f 1e fa             endbr64
 144:   55                      push   %rbp
 145:   48 89 e5                mov    %rsp,%rbp
 148:   48 8d 05 02 00 00 00    lea    0x2(%rip),%rax        # 151 <ibt_selftest_ip>
 14f:   ff e0                   jmp    *%rax

while the gcc version looks correct with

00000000000005c0 <ibt_selftest>:
 5c0:   f3 0f 1e fa             endbr64
 5c4:   48 8d 05 02 00 00 00    lea    0x2(%rip),%rax        # 5cd <ibt_selftest_ip>
 5cb:   ff e0                   jmp    *%rax

Attached as 0x1A63498B-traps.o.gz and 0x1A63498B-config.gz, the
hexadecimal number is the randconfig seed I used.

>> 49 vmlinux.o: warning: objtool: lkdtm_UNSET_SMEP+0x100: relocation to !ENDBR: native_write_cr4+0x4
>
> That one was on purpose I think.

Do you mean objtool intentionally warns, or the code intentionally does
it but objtool should ignore it? I would assume that only one of them
is right.

>> 29 mm/kasan/generic.o: warning: objtool: kasan_check_range+0x1e: call to addr_has_metadata() with UACCESS enabled
>> 29 mm/kasan/generic.o: warning: objtool: __asan_load2+0x11: call to addr_has_metadata() with UACCESS enabled
>
> I'm thinking addr_has_metadata() needs the __always_inline treatment, it
> seems to be present a lot.

Makes sense, I'll send a patch.

>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_out+0x15e: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_out+0x11d: stack state mismatch: cfa1=5+16 cfa2=4+8
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_in+0x150: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx_msg.o: warning: objtool: vmw_port_hb_in+0x13d: stack state mismatch: cfa1=5+16 cfa2=4+8
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_out+0xe2: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_out+0xd1: stack state mismatch: cfa1=5+16 cfa2=4+8
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_in+0xea: return with modified stack frame
>> 29 drivers/gpu/drm/vmwgfx/vmwgfx.o: warning: objtool: vmw_port_hb_in+0xcc: stack state mismatch: cfa1=5+16 cfa2=4+8
>
> Repro details?

last seen with gcc-13.0.1, but I think this happened with most older
versions already, attached 0x2D9ADB20-vmwgfx_msg.o

>> 15 vmlinux.o: warning: objtool: ftrace_likely_update+0xd2: call to __stack_chk_fail() with UACCESS enabled
>> 15 kernel/trace/trace_branch.o: warning: objtool: ftrace_likely_update+0x201: call to __stack_chk_fail() with UACCESS enabled
>
> -EWONTFIX, TRACE_BRANCH_PROFILING is fundamentally incompatible with
> lots of things.

I see CONFIG_X86_SMAP is now always enabled, does that mean that
TRACE_BRANCH_PROFILING no longer works at all on x86, or would it
still work most of the time?

>> 14 arch/x86/kvm/kvm.o: warning: objtool: .text+0x0: unreachable instruction
>
> details?

I see this with clang-14 through clang-16 or gcc. From an initial
analysis I see that virt/kvm/kvm_main.o contains empty __weak versions
of kvm_arch_mmu_notifier_invalidate_range and kvm_arch_guest_memory_reclaimed.

For some reason, after linking (with ld.bfd or ld.lld) the file into
arch/x86/kvm/kvm.o, the code for those two functions remains, but without
a symbol, which leads to a funny disassembly:

Disassembly of section .text:

0000000000000000 <kvm_is_zone_device_page-0x80>:
       0:       f3 0f 1e fa             endbr64
       4:       c3                      ret
       5:       66 66 66 66 66 66 2e    data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
       c:       0f 1f 84 00 00 00 00 
      13:       00 
      14:       66 66 66 66 66 66 2e    data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      1b:       0f 1f 84 00 00 00 00 
      22:       00 
      23:       66 66 66 66 66 66 2e    data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      2a:       0f 1f 84 00 00 00 00 
      31:       00 
      32:       66 66 66 66 66 2e 0f    data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      39:       1f 84 00 00 00 00 00 
      40:       f3 0f 1e fa             endbr64
      44:       c3                      ret
      45:       66 66 66 66 66 66 2e    data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      4c:       0f 1f 84 00 00 00 00 
      53:       00 
      54:       66 66 66 66 66 66 2e    data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      5b:       0f 1f 84 00 00 00 00 
      62:       00 
      63:       66 66 66 66 66 66 2e    data16 data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      6a:       0f 1f 84 00 00 00 00 
      71:       00 
      72:       66 66 66 66 66 2e 0f    data16 data16 data16 data16 cs nopw 0x0(%rax,%rax,1)
      79:       1f 84 00 00 00 00 00 

0000000000000080 <kvm_is_zone_device_page>:
      80:       f3 0f 1e fa             endbr64
....

When kvm_main.o is built with gcc, the two functions do not appear to
be part of the object file, but I don't see why not. 

Attaching 0x6BC495CE-kvm_main-gcc.o and 0x6BC495CE-kvm_main-clang.o

>> 2 mm/kmsan/report.o: warning: objtool: kmsan_report+0x0: call to __fentry__() with UACCESS enabled
>
> mm/kasan/Makefil:CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
>
> How?!?!
>
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_warning+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_poison_alloca+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_n+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_8+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_4+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_2+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_store_1+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_n+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_8+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_4+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_2+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_metadata_ptr_for_load_1+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_instrument_asm_store+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_get_context_state+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/instrumentation.o: warning: objtool: __msan_chain_origin+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/hooks.o: warning: objtool: kmsan_unpoison_memory+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/hooks.o: warning: objtool: kmsan_unpoison_entry_regs+0x0: call to __fentry__() with UACCESS enabled
>> 2 mm/kmsan/hooks.o: warning: objtool: kmsan_copy_to_user+0x0: call to __fentry__() with UACCESS enabled
>
> Something seems to be going badly wrong there... notrace is being lost.

It's kmsan, not kasan!

I'll send a patch to copy the relevant Makefile bits from kasan to kmsan:
--- a/mm/kmsan/Makefile
+++ b/mm/kmsan/Makefile
@@ -16,6 +16,14 @@ CC_FLAGS_KMSAN_RUNTIME += -DDISABLE_BRANCH_PROFILING
 
 CFLAGS_REMOVE.o = $(CC_FLAGS_FTRACE)
 
+# Disable ftrace to avoid recursion.
+CFLAGS_REMOVE_core.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_hooks.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_instrumentation.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE)
+
 CFLAGS_core.o := $(CC_FLAGS_KMSAN_RUNTIME)
 CFLAGS_hooks.o := $(CC_FLAGS_KMSAN_RUNTIME)
 CFLAGS_init.o := $(CC_FLAGS_KMSAN_RUNTIME)

     Arnd
Download attachment "0x1A63498B-config.gz" of type "application/gzip" (32661 bytes)

Download attachment "0x1A63498B-traps.o.gz" of type "application/gzip" (6323 bytes)

Download attachment "0x2D9ADB20-config.gz" of type "application/gzip" (45816 bytes)

Download attachment "0x2D9ADB20-vmwgfx_msg.o.gz" of type "application/gzip" (10528 bytes)

Download attachment "0x6BC495CE-kvm_main-clang.o" of type "application/x-object" (206032 bytes)

Download attachment "0x6BC495CE-kvm_main-gcc.o" of type "application/x-object" (190296 bytes)

Download attachment "0x6BC495CE-config" of type "application/octet-stream" (170474 bytes)

Powered by blists - more mailing lists