Message-ID: <CA+fCnZd6gXtwjBKSnChpT+dF9u8u68q2jVmR=MQHe4JhPBuvtw@mail.gmail.com>
Date: Mon, 23 Sep 2024 22:12:38 +0200
From: Andrey Konovalov <andreyknvl@...il.com>
To: Mark Rutland <mark.rutland@....com>, Alexander Potapenko <glider@...gle.com>
Cc: Marc Zyngier <maz@...nel.org>, Will Deacon <will@...nel.org>, 
	syzbot <syzbot+908886656a02769af987@...kaller.appspotmail.com>, 
	catalin.marinas@....com, linux-arm-kernel@...ts.infradead.org, 
	linux-kernel@...r.kernel.org, syzkaller-bugs@...glegroups.com, 
	kasan-dev <kasan-dev@...glegroups.com>, Aleksandr Nogikh <nogikh@...gle.com>, 
	Andrey Ryabinin <ryabinin.a.a@...il.com>
Subject: Re: [syzbot] [arm?] upstream test error: KASAN: invalid-access Write
 in setup_arch

On Mon, Sep 23, 2024 at 12:46 PM Mark Rutland <mark.rutland@....com> wrote:
>
> > > > ==================================================================
> > > > BUG: KASAN: invalid-access in smp_build_mpidr_hash arch/arm64/kernel/setup.c:133 [inline]
> > > > BUG: KASAN: invalid-access in setup_arch+0x984/0xd60 arch/arm64/kernel/setup.c:356
> > > > Write of size 4 at addr 03ff800086867e00 by task swapper/0
> > > > Pointer tag: [03], memory tag: [fe]
> > > >
> > > > CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.11.0-rc5-syzkaller-g33faa93bc856 #0
> > > > Hardware name: linux,dummy-virt (DT)
> > > > Call trace:
> > > >  dump_backtrace+0x204/0x3b8 arch/arm64/kernel/stacktrace.c:317
> > > >  show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:324
> > > >  __dump_stack lib/dump_stack.c:93 [inline]
> > > >  dump_stack_lvl+0x260/0x3b4 lib/dump_stack.c:119
> > > >  print_address_description mm/kasan/report.c:377 [inline]
> > > >  print_report+0x118/0x5ac mm/kasan/report.c:488
> > > >  kasan_report+0xc8/0x108 mm/kasan/report.c:601
> > > >  kasan_check_range+0x94/0xb8 mm/kasan/sw_tags.c:84
> > > >  __hwasan_store4_noabort+0x20/0x2c mm/kasan/sw_tags.c:149
> > > >  smp_build_mpidr_hash arch/arm64/kernel/setup.c:133 [inline]
> > > >  setup_arch+0x984/0xd60 arch/arm64/kernel/setup.c:356
> > > >  start_kernel+0xe0/0xff0 init/main.c:926
> > > >  __primary_switched+0x84/0x8c arch/arm64/kernel/head.S:243
> > > >
> > > > The buggy address belongs to stack of task swapper/0
> > > >
> > > > Memory state around the buggy address:
> > > >  ffff800086867c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > >  ffff800086867d00: 00 fe fe 00 00 00 fe fe fe fe fe fe fe fe fe fe
> > > > >ffff800086867e00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
> > > >                    ^
> > > >  ffff800086867f00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
> > > >  ffff800086868000: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe
> > > > ==================================================================
> > >
> > > I can't spot the issue here. We have a couple of fixed-length
> > > (4 element) arrays on the stack and they're indexed by a simple loop
> > > counter that runs from 0-3.
> >
> > Having trimmed the config to the extreme, I can only trigger the
> > warning with CONFIG_KASAN_SW_TAGS (CONFIG_KASAN_GENERIC does not
> > scream). Same thing if I use gcc 14.2.0.
> >
> > However, compiling with clang 14 (Debian clang version 14.0.6) does
> > *not* result in a screaming kernel, even with KASAN_SW_TAGS.

Yeah, this is #1 from https://bugzilla.kernel.org/show_bug.cgi?id=218854.

> > So I can see two possibilities here:
> >
> > - either gcc is incompatible with KASAN_SW_TAGS and the generic
> >   version is the only one that works
> >
> > - or we have a compiler bug on our hands.
> >
> > Frankly, I can't believe the later, as the code is so daft that I
> > can't imagine gcc getting it *that* wrong.
>
> It looks like what's happening here is:
>
> (1) With CONFIG_KASAN_SW_TAGS=y we pass the compiler
>     `-fsanitize=kernel-hwaddress`.
>
> (2) When GCC is passed `-fsanitize=hwaddress` or
>     `-fsanitize=kernel-hwaddress` it ignores
>     `__attribute__((no_sanitize_address))`, and instruments functions we
>     require are not instrumented.
>
>     I believe this is a compiler bug, as there doesn't seem to be a
>     separate attribute to prevent instrumentation in this mode.
>
> (3) In this config, smp_build_mpidr_hash() gets inlined into
>     setup_arch(), and as setup_arch() is instrumented, all of the stack
>     variables for smp_build_mpidr_hash() are initialized at the start of
>     setup_arch(), with calls to __hwasan_tag_memory().
>
>     At this point, we are using the early shadow (where a single page of
>     shadow is used for all memory).
>
> (4) In setup_arch(), we call kasan_init() to transition from the early
>     shadow to the runtime shadow. This replaces the early shadow memory
>     with new shadow memory initialized to KASAN_SHADOW_INIT (0xFE AKA
>     KASAN_TAG_INVALID), including the shadow for the stack.
>
> (5) Once the CPU returns back into setup_arch(), it's using the new
>     shadow initialized to 0xFE. Subsequent stack accesses which check
>     the shadow see 0xFE in the shadow, and fault. Note that in the dump
>     of the shadow above, the shadow around ffff800086867d80 and above is
>     all 0xFE, while below that functions have managed to clear the
>     shadow.
>
> Compiler test case below. Note that this demonstrates the compiler
> ignores  `__attribute__((no_sanitize_address))` regardless of
> KASAN_STACK, so KASAN_SW_TAGS is generally broken with GCC. All versions
> I tried were broken, from 11.3.0 to 14.2.0 inclusive.

Thank you for the detailed investigation report!

> I think we have to disable KASAN_SW_TAGS with GCC until this is fixed.

Sounds good to me.

Please reference https://bugzilla.kernel.org/show_bug.cgi?id=218854 if
you end up sending a patch for this.

Also the syzbot's kvm instance should probably be switched to Clang
(@Alexander).

Thank you!

Powered by blists - more mailing lists