Message-ID: <YoIOhZc3y11n5aMq@FVFF77S0Q05N>
Date:   Mon, 16 May 2022 09:42:45 +0100
From:   Mark Rutland <mark.rutland@....com>
To:     Masahiro Yamada <masahiroy@...nel.org>
Cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Vincent Mailhol <mailhol.vincent@...adoo.fr>,
        Will Deacon <will@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>
Subject: Re: [v2 2/2] kbuild: copy scripts/atomic/atomic-*.h to
 include/generated/atomic-*.h

[adding the atomics maintainers to Cc]

On Sat, May 14, 2022 at 01:02:07PM +0900, Masahiro Yamada wrote:
> include/linux/atomic/*.h are generated by scripts/atomic/gen-atomics.sh.
> 
> To ensure they are not edited, scripts/atomic/check-atomics.sh checks
> sha1sum every time.
> 
> This commit moves include/linux/atomic/*.h to scripts/atomic/*.h_shipped,
> which are copied to include/generated/ at build time:
> 
>   COPY    include/generated/atomic-instrumented.h
>   COPY    include/generated/atomic-long.h
>   COPY    include/generated/atomic-arch-fallback.h

FWIW, moving these and copying them at build time is fine by me, given that
better indicates that these are generated.

> I dropped the sha1sum checks. I hope nobody would try to directly modify
> *_shipped files.

I'd prefer to keep the sha1sum check, because we're worried that people *will*
modify them, and that won't be noticed until the next time they're regenerated.

> Kbuild runs more and more tools at build time these days because they
> are fast enough on modern systems.
> 
> For example,
> 
>  - 29c833061c1d8c2d1d23a62e7061561eadd76cdb
>    ("kconfig: generate lexer and parser during build instead of shipping")
> 
>  - 7c0303ff7e67b637c47d8afee533ca9e2a02359b
>    ("crypto: arm - generate *.S by Perl at build time instead of shipping them")
> 
> Yet, gen-atomics.sh is too slow.

Yes; we'd originally wanted to run them at build time, but that was too slow,
and as a compromise we moved to regenerating them whenever we changed the
scripting.

> I guess it can be improved because it is just simple text processing.
> Then, Kbuild can execute it at build time.

It is in theory possible to make them much faster, yes. The actual slowness of
the scripting is largely due to using sub-shells resulting in a load of
fork+exec, which could be avoided if using a real scripting language.

Practically speaking though, we're rather limited in what we can rely upon
being available. We chose to use POSIX shell as a lowest-common-demoninator,
and anything that'd be *nice* to use isn't going to always be available,
meaning that even if we make it faster we can't necessarily build it all the
time anyway.

Thanks,
Mark.

> 
> Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> ---
>  Kbuild                                        |   26 +-
>  include/linux/atomic/atomic-arch-fallback.h   | 2391 +----------------
>  include/linux/atomic/atomic-instrumented.h    | 2050 +-------------
>  include/linux/atomic/atomic-long.h            | 1016 +------
>  .../atomic/atomic-arch-fallback.h_shipped     |    1 -
>  .../atomic/atomic-instrumented.h_shipped      |    1 -
>  .../atomic/atomic-long.h_shipped              |    1 -
>  scripts/atomic/check-atomics.sh               |   33 -
>  scripts/atomic/gen-atomics.sh                 |   14 +-
>  9 files changed, 24 insertions(+), 5509 deletions(-)
>  copy include/linux/atomic/atomic-arch-fallback.h => scripts/atomic/atomic-arch-fallback.h_shipped (99%)
>  copy include/linux/atomic/atomic-instrumented.h => scripts/atomic/atomic-instrumented.h_shipped (99%)
>  copy include/linux/atomic/atomic-long.h => scripts/atomic/atomic-long.h_shipped (99%)
>  delete mode 100755 scripts/atomic/check-atomics.sh
> 
> diff --git a/Kbuild b/Kbuild
> index fa441b98c9f6..768921bca727 100644
> --- a/Kbuild
> +++ b/Kbuild
> @@ -2,13 +2,24 @@
>  #
>  # Kbuild for top-level directory of the kernel
>  
> +####
> +# Copy atomic headers
> +
> +atomic-files := $(addprefix include/generated/, \
> +		atomic-instrumented.h atomic-long.h atomic-arch-fallback.h)
> +
> +targets := $(atomic-files)
> +
> +$(atomic-files): include/generated/%.h: $(srctree)/scripts/atomic/%.h_shipped
> +	$(call cmd,copy)
> +
>  #####
>  # Generate bounds.h
>  
>  bounds-file := include/generated/bounds.h
>  
>  always-y := $(bounds-file)
> -targets := kernel/bounds.s
> +targets += kernel/bounds.s
>  
>  $(bounds-file): kernel/bounds.s FORCE
>  	$(call filechk,offsets,__LINUX_BOUNDS_H__)
> @@ -31,7 +42,7 @@ offsets-file := include/generated/asm-offsets.h
>  always-y += $(offsets-file)
>  targets += arch/$(SRCARCH)/kernel/asm-offsets.s
>  
> -arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file)
> +arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file) $(atomic-files)
>  
>  $(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s FORCE
>  	$(call filechk,offsets,__ASM_OFFSETS_H__)
> @@ -46,14 +57,3 @@ quiet_cmd_syscalls = CALL    $<
>  
>  missing-syscalls: scripts/checksyscalls.sh $(offsets-file) FORCE
>  	$(call cmd,syscalls)
> -
> -#####
> -# Check atomic headers are up-to-date
> -
> -always-y += old-atomics
> -
> -quiet_cmd_atomics = CALL    $<
> -      cmd_atomics = $(CONFIG_SHELL) $<
> -
> -old-atomics: scripts/atomic/check-atomics.sh FORCE
> -	$(call cmd,atomics)
> diff --git a/include/linux/atomic/atomic-arch-fallback.h b/include/linux/atomic/atomic-arch-fallback.h
> index 6db58d180866..b13148ad6bd6 100644
> --- a/include/linux/atomic/atomic-arch-fallback.h
> +++ b/include/linux/atomic/atomic-arch-fallback.h
> @@ -1,2389 +1,2 @@
> -// SPDX-License-Identifier: GPL-2.0
> -
> -// Generated by scripts/atomic/gen-atomic-fallback.sh
> -// DO NOT MODIFY THIS FILE DIRECTLY
> -
> -#ifndef _LINUX_ATOMIC_FALLBACK_H
> -#define _LINUX_ATOMIC_FALLBACK_H
> -
> -#include <linux/compiler.h>
> -
> -#ifndef arch_xchg_relaxed
> -#define arch_xchg_acquire arch_xchg
> -#define arch_xchg_release arch_xchg
> -#define arch_xchg_relaxed arch_xchg
> -#else /* arch_xchg_relaxed */
> -
> -#ifndef arch_xchg_acquire
> -#define arch_xchg_acquire(...) \
> -	__atomic_op_acquire(arch_xchg, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_xchg_release
> -#define arch_xchg_release(...) \
> -	__atomic_op_release(arch_xchg, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_xchg
> -#define arch_xchg(...) \
> -	__atomic_op_fence(arch_xchg, __VA_ARGS__)
> -#endif
> -
> -#endif /* arch_xchg_relaxed */
> -
> -#ifndef arch_cmpxchg_relaxed
> -#define arch_cmpxchg_acquire arch_cmpxchg
> -#define arch_cmpxchg_release arch_cmpxchg
> -#define arch_cmpxchg_relaxed arch_cmpxchg
> -#else /* arch_cmpxchg_relaxed */
> -
> -#ifndef arch_cmpxchg_acquire
> -#define arch_cmpxchg_acquire(...) \
> -	__atomic_op_acquire(arch_cmpxchg, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_cmpxchg_release
> -#define arch_cmpxchg_release(...) \
> -	__atomic_op_release(arch_cmpxchg, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_cmpxchg
> -#define arch_cmpxchg(...) \
> -	__atomic_op_fence(arch_cmpxchg, __VA_ARGS__)
> -#endif
> -
> -#endif /* arch_cmpxchg_relaxed */
> -
> -#ifndef arch_cmpxchg64_relaxed
> -#define arch_cmpxchg64_acquire arch_cmpxchg64
> -#define arch_cmpxchg64_release arch_cmpxchg64
> -#define arch_cmpxchg64_relaxed arch_cmpxchg64
> -#else /* arch_cmpxchg64_relaxed */
> -
> -#ifndef arch_cmpxchg64_acquire
> -#define arch_cmpxchg64_acquire(...) \
> -	__atomic_op_acquire(arch_cmpxchg64, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_cmpxchg64_release
> -#define arch_cmpxchg64_release(...) \
> -	__atomic_op_release(arch_cmpxchg64, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_cmpxchg64
> -#define arch_cmpxchg64(...) \
> -	__atomic_op_fence(arch_cmpxchg64, __VA_ARGS__)
> -#endif
> -
> -#endif /* arch_cmpxchg64_relaxed */
> -
> -#ifndef arch_try_cmpxchg_relaxed
> -#ifdef arch_try_cmpxchg
> -#define arch_try_cmpxchg_acquire arch_try_cmpxchg
> -#define arch_try_cmpxchg_release arch_try_cmpxchg
> -#define arch_try_cmpxchg_relaxed arch_try_cmpxchg
> -#endif /* arch_try_cmpxchg */
> -
> -#ifndef arch_try_cmpxchg
> -#define arch_try_cmpxchg(_ptr, _oldp, _new) \
> -({ \
> -	typeof(*(_ptr)) *___op = (_oldp), ___o = *___op, ___r; \
> -	___r = arch_cmpxchg((_ptr), ___o, (_new)); \
> -	if (unlikely(___r != ___o)) \
> -		*___op = ___r; \
> -	likely(___r == ___o); \
> -})
> -#endif /* arch_try_cmpxchg */
> -
> -#ifndef arch_try_cmpxchg_acquire
> -#define arch_try_cmpxchg_acquire(_ptr, _oldp, _new) \
> -({ \
> -	typeof(*(_ptr)) *___op = (_oldp), ___o = *___op, ___r; \
> -	___r = arch_cmpxchg_acquire((_ptr), ___o, (_new)); \
> -	if (unlikely(___r != ___o)) \
> -		*___op = ___r; \
> -	likely(___r == ___o); \
> -})
> -#endif /* arch_try_cmpxchg_acquire */
> -
> -#ifndef arch_try_cmpxchg_release
> -#define arch_try_cmpxchg_release(_ptr, _oldp, _new) \
> -({ \
> -	typeof(*(_ptr)) *___op = (_oldp), ___o = *___op, ___r; \
> -	___r = arch_cmpxchg_release((_ptr), ___o, (_new)); \
> -	if (unlikely(___r != ___o)) \
> -		*___op = ___r; \
> -	likely(___r == ___o); \
> -})
> -#endif /* arch_try_cmpxchg_release */
> -
> -#ifndef arch_try_cmpxchg_relaxed
> -#define arch_try_cmpxchg_relaxed(_ptr, _oldp, _new) \
> -({ \
> -	typeof(*(_ptr)) *___op = (_oldp), ___o = *___op, ___r; \
> -	___r = arch_cmpxchg_relaxed((_ptr), ___o, (_new)); \
> -	if (unlikely(___r != ___o)) \
> -		*___op = ___r; \
> -	likely(___r == ___o); \
> -})
> -#endif /* arch_try_cmpxchg_relaxed */
> -
> -#else /* arch_try_cmpxchg_relaxed */
> -
> -#ifndef arch_try_cmpxchg_acquire
> -#define arch_try_cmpxchg_acquire(...) \
> -	__atomic_op_acquire(arch_try_cmpxchg, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_try_cmpxchg_release
> -#define arch_try_cmpxchg_release(...) \
> -	__atomic_op_release(arch_try_cmpxchg, __VA_ARGS__)
> -#endif
> -
> -#ifndef arch_try_cmpxchg
> -#define arch_try_cmpxchg(...) \
> -	__atomic_op_fence(arch_try_cmpxchg, __VA_ARGS__)
> -#endif
> -
> -#endif /* arch_try_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic_read_acquire
> -static __always_inline int
> -arch_atomic_read_acquire(const atomic_t *v)
> -{
> -	int ret;
> -
> -	if (__native_word(atomic_t)) {
> -		ret = smp_load_acquire(&(v)->counter);
> -	} else {
> -		ret = arch_atomic_read(v);
> -		__atomic_acquire_fence();
> -	}
> -
> -	return ret;
> -}
> -#define arch_atomic_read_acquire arch_atomic_read_acquire
> -#endif
> -
> -#ifndef arch_atomic_set_release
> -static __always_inline void
> -arch_atomic_set_release(atomic_t *v, int i)
> -{
> -	if (__native_word(atomic_t)) {
> -		smp_store_release(&(v)->counter, i);
> -	} else {
> -		__atomic_release_fence();
> -		arch_atomic_set(v, i);
> -	}
> -}
> -#define arch_atomic_set_release arch_atomic_set_release
> -#endif
> -
> -#ifndef arch_atomic_add_return_relaxed
> -#define arch_atomic_add_return_acquire arch_atomic_add_return
> -#define arch_atomic_add_return_release arch_atomic_add_return
> -#define arch_atomic_add_return_relaxed arch_atomic_add_return
> -#else /* arch_atomic_add_return_relaxed */
> -
> -#ifndef arch_atomic_add_return_acquire
> -static __always_inline int
> -arch_atomic_add_return_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_add_return_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_add_return_acquire arch_atomic_add_return_acquire
> -#endif
> -
> -#ifndef arch_atomic_add_return_release
> -static __always_inline int
> -arch_atomic_add_return_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_add_return_relaxed(i, v);
> -}
> -#define arch_atomic_add_return_release arch_atomic_add_return_release
> -#endif
> -
> -#ifndef arch_atomic_add_return
> -static __always_inline int
> -arch_atomic_add_return(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_add_return_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_add_return arch_atomic_add_return
> -#endif
> -
> -#endif /* arch_atomic_add_return_relaxed */
> -
> -#ifndef arch_atomic_fetch_add_relaxed
> -#define arch_atomic_fetch_add_acquire arch_atomic_fetch_add
> -#define arch_atomic_fetch_add_release arch_atomic_fetch_add
> -#define arch_atomic_fetch_add_relaxed arch_atomic_fetch_add
> -#else /* arch_atomic_fetch_add_relaxed */
> -
> -#ifndef arch_atomic_fetch_add_acquire
> -static __always_inline int
> -arch_atomic_fetch_add_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_add_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_add_acquire arch_atomic_fetch_add_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_add_release
> -static __always_inline int
> -arch_atomic_fetch_add_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_add_relaxed(i, v);
> -}
> -#define arch_atomic_fetch_add_release arch_atomic_fetch_add_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_add
> -static __always_inline int
> -arch_atomic_fetch_add(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_add_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_add arch_atomic_fetch_add
> -#endif
> -
> -#endif /* arch_atomic_fetch_add_relaxed */
> -
> -#ifndef arch_atomic_sub_return_relaxed
> -#define arch_atomic_sub_return_acquire arch_atomic_sub_return
> -#define arch_atomic_sub_return_release arch_atomic_sub_return
> -#define arch_atomic_sub_return_relaxed arch_atomic_sub_return
> -#else /* arch_atomic_sub_return_relaxed */
> -
> -#ifndef arch_atomic_sub_return_acquire
> -static __always_inline int
> -arch_atomic_sub_return_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_sub_return_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_sub_return_acquire arch_atomic_sub_return_acquire
> -#endif
> -
> -#ifndef arch_atomic_sub_return_release
> -static __always_inline int
> -arch_atomic_sub_return_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_sub_return_relaxed(i, v);
> -}
> -#define arch_atomic_sub_return_release arch_atomic_sub_return_release
> -#endif
> -
> -#ifndef arch_atomic_sub_return
> -static __always_inline int
> -arch_atomic_sub_return(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_sub_return_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_sub_return arch_atomic_sub_return
> -#endif
> -
> -#endif /* arch_atomic_sub_return_relaxed */
> -
> -#ifndef arch_atomic_fetch_sub_relaxed
> -#define arch_atomic_fetch_sub_acquire arch_atomic_fetch_sub
> -#define arch_atomic_fetch_sub_release arch_atomic_fetch_sub
> -#define arch_atomic_fetch_sub_relaxed arch_atomic_fetch_sub
> -#else /* arch_atomic_fetch_sub_relaxed */
> -
> -#ifndef arch_atomic_fetch_sub_acquire
> -static __always_inline int
> -arch_atomic_fetch_sub_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_sub_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_sub_acquire arch_atomic_fetch_sub_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_sub_release
> -static __always_inline int
> -arch_atomic_fetch_sub_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_sub_relaxed(i, v);
> -}
> -#define arch_atomic_fetch_sub_release arch_atomic_fetch_sub_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_sub
> -static __always_inline int
> -arch_atomic_fetch_sub(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_sub_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_sub arch_atomic_fetch_sub
> -#endif
> -
> -#endif /* arch_atomic_fetch_sub_relaxed */
> -
> -#ifndef arch_atomic_inc
> -static __always_inline void
> -arch_atomic_inc(atomic_t *v)
> -{
> -	arch_atomic_add(1, v);
> -}
> -#define arch_atomic_inc arch_atomic_inc
> -#endif
> -
> -#ifndef arch_atomic_inc_return_relaxed
> -#ifdef arch_atomic_inc_return
> -#define arch_atomic_inc_return_acquire arch_atomic_inc_return
> -#define arch_atomic_inc_return_release arch_atomic_inc_return
> -#define arch_atomic_inc_return_relaxed arch_atomic_inc_return
> -#endif /* arch_atomic_inc_return */
> -
> -#ifndef arch_atomic_inc_return
> -static __always_inline int
> -arch_atomic_inc_return(atomic_t *v)
> -{
> -	return arch_atomic_add_return(1, v);
> -}
> -#define arch_atomic_inc_return arch_atomic_inc_return
> -#endif
> -
> -#ifndef arch_atomic_inc_return_acquire
> -static __always_inline int
> -arch_atomic_inc_return_acquire(atomic_t *v)
> -{
> -	return arch_atomic_add_return_acquire(1, v);
> -}
> -#define arch_atomic_inc_return_acquire arch_atomic_inc_return_acquire
> -#endif
> -
> -#ifndef arch_atomic_inc_return_release
> -static __always_inline int
> -arch_atomic_inc_return_release(atomic_t *v)
> -{
> -	return arch_atomic_add_return_release(1, v);
> -}
> -#define arch_atomic_inc_return_release arch_atomic_inc_return_release
> -#endif
> -
> -#ifndef arch_atomic_inc_return_relaxed
> -static __always_inline int
> -arch_atomic_inc_return_relaxed(atomic_t *v)
> -{
> -	return arch_atomic_add_return_relaxed(1, v);
> -}
> -#define arch_atomic_inc_return_relaxed arch_atomic_inc_return_relaxed
> -#endif
> -
> -#else /* arch_atomic_inc_return_relaxed */
> -
> -#ifndef arch_atomic_inc_return_acquire
> -static __always_inline int
> -arch_atomic_inc_return_acquire(atomic_t *v)
> -{
> -	int ret = arch_atomic_inc_return_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_inc_return_acquire arch_atomic_inc_return_acquire
> -#endif
> -
> -#ifndef arch_atomic_inc_return_release
> -static __always_inline int
> -arch_atomic_inc_return_release(atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_inc_return_relaxed(v);
> -}
> -#define arch_atomic_inc_return_release arch_atomic_inc_return_release
> -#endif
> -
> -#ifndef arch_atomic_inc_return
> -static __always_inline int
> -arch_atomic_inc_return(atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_inc_return_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_inc_return arch_atomic_inc_return
> -#endif
> -
> -#endif /* arch_atomic_inc_return_relaxed */
> -
> -#ifndef arch_atomic_fetch_inc_relaxed
> -#ifdef arch_atomic_fetch_inc
> -#define arch_atomic_fetch_inc_acquire arch_atomic_fetch_inc
> -#define arch_atomic_fetch_inc_release arch_atomic_fetch_inc
> -#define arch_atomic_fetch_inc_relaxed arch_atomic_fetch_inc
> -#endif /* arch_atomic_fetch_inc */
> -
> -#ifndef arch_atomic_fetch_inc
> -static __always_inline int
> -arch_atomic_fetch_inc(atomic_t *v)
> -{
> -	return arch_atomic_fetch_add(1, v);
> -}
> -#define arch_atomic_fetch_inc arch_atomic_fetch_inc
> -#endif
> -
> -#ifndef arch_atomic_fetch_inc_acquire
> -static __always_inline int
> -arch_atomic_fetch_inc_acquire(atomic_t *v)
> -{
> -	return arch_atomic_fetch_add_acquire(1, v);
> -}
> -#define arch_atomic_fetch_inc_acquire arch_atomic_fetch_inc_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_inc_release
> -static __always_inline int
> -arch_atomic_fetch_inc_release(atomic_t *v)
> -{
> -	return arch_atomic_fetch_add_release(1, v);
> -}
> -#define arch_atomic_fetch_inc_release arch_atomic_fetch_inc_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_inc_relaxed
> -static __always_inline int
> -arch_atomic_fetch_inc_relaxed(atomic_t *v)
> -{
> -	return arch_atomic_fetch_add_relaxed(1, v);
> -}
> -#define arch_atomic_fetch_inc_relaxed arch_atomic_fetch_inc_relaxed
> -#endif
> -
> -#else /* arch_atomic_fetch_inc_relaxed */
> -
> -#ifndef arch_atomic_fetch_inc_acquire
> -static __always_inline int
> -arch_atomic_fetch_inc_acquire(atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_inc_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_inc_acquire arch_atomic_fetch_inc_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_inc_release
> -static __always_inline int
> -arch_atomic_fetch_inc_release(atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_inc_relaxed(v);
> -}
> -#define arch_atomic_fetch_inc_release arch_atomic_fetch_inc_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_inc
> -static __always_inline int
> -arch_atomic_fetch_inc(atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_inc_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_inc arch_atomic_fetch_inc
> -#endif
> -
> -#endif /* arch_atomic_fetch_inc_relaxed */
> -
> -#ifndef arch_atomic_dec
> -static __always_inline void
> -arch_atomic_dec(atomic_t *v)
> -{
> -	arch_atomic_sub(1, v);
> -}
> -#define arch_atomic_dec arch_atomic_dec
> -#endif
> -
> -#ifndef arch_atomic_dec_return_relaxed
> -#ifdef arch_atomic_dec_return
> -#define arch_atomic_dec_return_acquire arch_atomic_dec_return
> -#define arch_atomic_dec_return_release arch_atomic_dec_return
> -#define arch_atomic_dec_return_relaxed arch_atomic_dec_return
> -#endif /* arch_atomic_dec_return */
> -
> -#ifndef arch_atomic_dec_return
> -static __always_inline int
> -arch_atomic_dec_return(atomic_t *v)
> -{
> -	return arch_atomic_sub_return(1, v);
> -}
> -#define arch_atomic_dec_return arch_atomic_dec_return
> -#endif
> -
> -#ifndef arch_atomic_dec_return_acquire
> -static __always_inline int
> -arch_atomic_dec_return_acquire(atomic_t *v)
> -{
> -	return arch_atomic_sub_return_acquire(1, v);
> -}
> -#define arch_atomic_dec_return_acquire arch_atomic_dec_return_acquire
> -#endif
> -
> -#ifndef arch_atomic_dec_return_release
> -static __always_inline int
> -arch_atomic_dec_return_release(atomic_t *v)
> -{
> -	return arch_atomic_sub_return_release(1, v);
> -}
> -#define arch_atomic_dec_return_release arch_atomic_dec_return_release
> -#endif
> -
> -#ifndef arch_atomic_dec_return_relaxed
> -static __always_inline int
> -arch_atomic_dec_return_relaxed(atomic_t *v)
> -{
> -	return arch_atomic_sub_return_relaxed(1, v);
> -}
> -#define arch_atomic_dec_return_relaxed arch_atomic_dec_return_relaxed
> -#endif
> -
> -#else /* arch_atomic_dec_return_relaxed */
> -
> -#ifndef arch_atomic_dec_return_acquire
> -static __always_inline int
> -arch_atomic_dec_return_acquire(atomic_t *v)
> -{
> -	int ret = arch_atomic_dec_return_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_dec_return_acquire arch_atomic_dec_return_acquire
> -#endif
> -
> -#ifndef arch_atomic_dec_return_release
> -static __always_inline int
> -arch_atomic_dec_return_release(atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_dec_return_relaxed(v);
> -}
> -#define arch_atomic_dec_return_release arch_atomic_dec_return_release
> -#endif
> -
> -#ifndef arch_atomic_dec_return
> -static __always_inline int
> -arch_atomic_dec_return(atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_dec_return_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_dec_return arch_atomic_dec_return
> -#endif
> -
> -#endif /* arch_atomic_dec_return_relaxed */
> -
> -#ifndef arch_atomic_fetch_dec_relaxed
> -#ifdef arch_atomic_fetch_dec
> -#define arch_atomic_fetch_dec_acquire arch_atomic_fetch_dec
> -#define arch_atomic_fetch_dec_release arch_atomic_fetch_dec
> -#define arch_atomic_fetch_dec_relaxed arch_atomic_fetch_dec
> -#endif /* arch_atomic_fetch_dec */
> -
> -#ifndef arch_atomic_fetch_dec
> -static __always_inline int
> -arch_atomic_fetch_dec(atomic_t *v)
> -{
> -	return arch_atomic_fetch_sub(1, v);
> -}
> -#define arch_atomic_fetch_dec arch_atomic_fetch_dec
> -#endif
> -
> -#ifndef arch_atomic_fetch_dec_acquire
> -static __always_inline int
> -arch_atomic_fetch_dec_acquire(atomic_t *v)
> -{
> -	return arch_atomic_fetch_sub_acquire(1, v);
> -}
> -#define arch_atomic_fetch_dec_acquire arch_atomic_fetch_dec_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_dec_release
> -static __always_inline int
> -arch_atomic_fetch_dec_release(atomic_t *v)
> -{
> -	return arch_atomic_fetch_sub_release(1, v);
> -}
> -#define arch_atomic_fetch_dec_release arch_atomic_fetch_dec_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_dec_relaxed
> -static __always_inline int
> -arch_atomic_fetch_dec_relaxed(atomic_t *v)
> -{
> -	return arch_atomic_fetch_sub_relaxed(1, v);
> -}
> -#define arch_atomic_fetch_dec_relaxed arch_atomic_fetch_dec_relaxed
> -#endif
> -
> -#else /* arch_atomic_fetch_dec_relaxed */
> -
> -#ifndef arch_atomic_fetch_dec_acquire
> -static __always_inline int
> -arch_atomic_fetch_dec_acquire(atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_dec_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_dec_acquire arch_atomic_fetch_dec_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_dec_release
> -static __always_inline int
> -arch_atomic_fetch_dec_release(atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_dec_relaxed(v);
> -}
> -#define arch_atomic_fetch_dec_release arch_atomic_fetch_dec_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_dec
> -static __always_inline int
> -arch_atomic_fetch_dec(atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_dec_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_dec arch_atomic_fetch_dec
> -#endif
> -
> -#endif /* arch_atomic_fetch_dec_relaxed */
> -
> -#ifndef arch_atomic_fetch_and_relaxed
> -#define arch_atomic_fetch_and_acquire arch_atomic_fetch_and
> -#define arch_atomic_fetch_and_release arch_atomic_fetch_and
> -#define arch_atomic_fetch_and_relaxed arch_atomic_fetch_and
> -#else /* arch_atomic_fetch_and_relaxed */
> -
> -#ifndef arch_atomic_fetch_and_acquire
> -static __always_inline int
> -arch_atomic_fetch_and_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_and_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_and_acquire arch_atomic_fetch_and_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_and_release
> -static __always_inline int
> -arch_atomic_fetch_and_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_and_relaxed(i, v);
> -}
> -#define arch_atomic_fetch_and_release arch_atomic_fetch_and_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_and
> -static __always_inline int
> -arch_atomic_fetch_and(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_and_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_and arch_atomic_fetch_and
> -#endif
> -
> -#endif /* arch_atomic_fetch_and_relaxed */
> -
> -#ifndef arch_atomic_andnot
> -static __always_inline void
> -arch_atomic_andnot(int i, atomic_t *v)
> -{
> -	arch_atomic_and(~i, v);
> -}
> -#define arch_atomic_andnot arch_atomic_andnot
> -#endif
> -
> -#ifndef arch_atomic_fetch_andnot_relaxed
> -#ifdef arch_atomic_fetch_andnot
> -#define arch_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot
> -#define arch_atomic_fetch_andnot_release arch_atomic_fetch_andnot
> -#define arch_atomic_fetch_andnot_relaxed arch_atomic_fetch_andnot
> -#endif /* arch_atomic_fetch_andnot */
> -
> -#ifndef arch_atomic_fetch_andnot
> -static __always_inline int
> -arch_atomic_fetch_andnot(int i, atomic_t *v)
> -{
> -	return arch_atomic_fetch_and(~i, v);
> -}
> -#define arch_atomic_fetch_andnot arch_atomic_fetch_andnot
> -#endif
> -
> -#ifndef arch_atomic_fetch_andnot_acquire
> -static __always_inline int
> -arch_atomic_fetch_andnot_acquire(int i, atomic_t *v)
> -{
> -	return arch_atomic_fetch_and_acquire(~i, v);
> -}
> -#define arch_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_andnot_release
> -static __always_inline int
> -arch_atomic_fetch_andnot_release(int i, atomic_t *v)
> -{
> -	return arch_atomic_fetch_and_release(~i, v);
> -}
> -#define arch_atomic_fetch_andnot_release arch_atomic_fetch_andnot_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_andnot_relaxed
> -static __always_inline int
> -arch_atomic_fetch_andnot_relaxed(int i, atomic_t *v)
> -{
> -	return arch_atomic_fetch_and_relaxed(~i, v);
> -}
> -#define arch_atomic_fetch_andnot_relaxed arch_atomic_fetch_andnot_relaxed
> -#endif
> -
> -#else /* arch_atomic_fetch_andnot_relaxed */
> -
> -#ifndef arch_atomic_fetch_andnot_acquire
> -static __always_inline int
> -arch_atomic_fetch_andnot_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_andnot_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_andnot_acquire arch_atomic_fetch_andnot_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_andnot_release
> -static __always_inline int
> -arch_atomic_fetch_andnot_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_andnot_relaxed(i, v);
> -}
> -#define arch_atomic_fetch_andnot_release arch_atomic_fetch_andnot_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_andnot
> -static __always_inline int
> -arch_atomic_fetch_andnot(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_andnot_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_andnot arch_atomic_fetch_andnot
> -#endif
> -
> -#endif /* arch_atomic_fetch_andnot_relaxed */
> -
> -#ifndef arch_atomic_fetch_or_relaxed
> -#define arch_atomic_fetch_or_acquire arch_atomic_fetch_or
> -#define arch_atomic_fetch_or_release arch_atomic_fetch_or
> -#define arch_atomic_fetch_or_relaxed arch_atomic_fetch_or
> -#else /* arch_atomic_fetch_or_relaxed */
> -
> -#ifndef arch_atomic_fetch_or_acquire
> -static __always_inline int
> -arch_atomic_fetch_or_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_or_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_or_acquire arch_atomic_fetch_or_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_or_release
> -static __always_inline int
> -arch_atomic_fetch_or_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_or_relaxed(i, v);
> -}
> -#define arch_atomic_fetch_or_release arch_atomic_fetch_or_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_or
> -static __always_inline int
> -arch_atomic_fetch_or(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_or_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_or arch_atomic_fetch_or
> -#endif
> -
> -#endif /* arch_atomic_fetch_or_relaxed */
> -
> -#ifndef arch_atomic_fetch_xor_relaxed
> -#define arch_atomic_fetch_xor_acquire arch_atomic_fetch_xor
> -#define arch_atomic_fetch_xor_release arch_atomic_fetch_xor
> -#define arch_atomic_fetch_xor_relaxed arch_atomic_fetch_xor
> -#else /* arch_atomic_fetch_xor_relaxed */
> -
> -#ifndef arch_atomic_fetch_xor_acquire
> -static __always_inline int
> -arch_atomic_fetch_xor_acquire(int i, atomic_t *v)
> -{
> -	int ret = arch_atomic_fetch_xor_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_xor_acquire arch_atomic_fetch_xor_acquire
> -#endif
> -
> -#ifndef arch_atomic_fetch_xor_release
> -static __always_inline int
> -arch_atomic_fetch_xor_release(int i, atomic_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_fetch_xor_relaxed(i, v);
> -}
> -#define arch_atomic_fetch_xor_release arch_atomic_fetch_xor_release
> -#endif
> -
> -#ifndef arch_atomic_fetch_xor
> -static __always_inline int
> -arch_atomic_fetch_xor(int i, atomic_t *v)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_fetch_xor_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_fetch_xor arch_atomic_fetch_xor
> -#endif
> -
> -#endif /* arch_atomic_fetch_xor_relaxed */
> -
> -#ifndef arch_atomic_xchg_relaxed
> -#define arch_atomic_xchg_acquire arch_atomic_xchg
> -#define arch_atomic_xchg_release arch_atomic_xchg
> -#define arch_atomic_xchg_relaxed arch_atomic_xchg
> -#else /* arch_atomic_xchg_relaxed */
> -
> -#ifndef arch_atomic_xchg_acquire
> -static __always_inline int
> -arch_atomic_xchg_acquire(atomic_t *v, int i)
> -{
> -	int ret = arch_atomic_xchg_relaxed(v, i);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_xchg_acquire arch_atomic_xchg_acquire
> -#endif
> -
> -#ifndef arch_atomic_xchg_release
> -static __always_inline int
> -arch_atomic_xchg_release(atomic_t *v, int i)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_xchg_relaxed(v, i);
> -}
> -#define arch_atomic_xchg_release arch_atomic_xchg_release
> -#endif
> -
> -#ifndef arch_atomic_xchg
> -static __always_inline int
> -arch_atomic_xchg(atomic_t *v, int i)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_xchg_relaxed(v, i);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_xchg arch_atomic_xchg
> -#endif
> -
> -#endif /* arch_atomic_xchg_relaxed */
> -
> -#ifndef arch_atomic_cmpxchg_relaxed
> -#define arch_atomic_cmpxchg_acquire arch_atomic_cmpxchg
> -#define arch_atomic_cmpxchg_release arch_atomic_cmpxchg
> -#define arch_atomic_cmpxchg_relaxed arch_atomic_cmpxchg
> -#else /* arch_atomic_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic_cmpxchg_acquire
> -static __always_inline int
> -arch_atomic_cmpxchg_acquire(atomic_t *v, int old, int new)
> -{
> -	int ret = arch_atomic_cmpxchg_relaxed(v, old, new);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_cmpxchg_acquire arch_atomic_cmpxchg_acquire
> -#endif
> -
> -#ifndef arch_atomic_cmpxchg_release
> -static __always_inline int
> -arch_atomic_cmpxchg_release(atomic_t *v, int old, int new)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_cmpxchg_relaxed(v, old, new);
> -}
> -#define arch_atomic_cmpxchg_release arch_atomic_cmpxchg_release
> -#endif
> -
> -#ifndef arch_atomic_cmpxchg
> -static __always_inline int
> -arch_atomic_cmpxchg(atomic_t *v, int old, int new)
> -{
> -	int ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_cmpxchg_relaxed(v, old, new);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_cmpxchg arch_atomic_cmpxchg
> -#endif
> -
> -#endif /* arch_atomic_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic_try_cmpxchg_relaxed
> -#ifdef arch_atomic_try_cmpxchg
> -#define arch_atomic_try_cmpxchg_acquire arch_atomic_try_cmpxchg
> -#define arch_atomic_try_cmpxchg_release arch_atomic_try_cmpxchg
> -#define arch_atomic_try_cmpxchg_relaxed arch_atomic_try_cmpxchg
> -#endif /* arch_atomic_try_cmpxchg */
> -
> -#ifndef arch_atomic_try_cmpxchg
> -static __always_inline bool
> -arch_atomic_try_cmpxchg(atomic_t *v, int *old, int new)
> -{
> -	int r, o = *old;
> -	r = arch_atomic_cmpxchg(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic_try_cmpxchg arch_atomic_try_cmpxchg
> -#endif
> -
> -#ifndef arch_atomic_try_cmpxchg_acquire
> -static __always_inline bool
> -arch_atomic_try_cmpxchg_acquire(atomic_t *v, int *old, int new)
> -{
> -	int r, o = *old;
> -	r = arch_atomic_cmpxchg_acquire(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic_try_cmpxchg_acquire arch_atomic_try_cmpxchg_acquire
> -#endif
> -
> -#ifndef arch_atomic_try_cmpxchg_release
> -static __always_inline bool
> -arch_atomic_try_cmpxchg_release(atomic_t *v, int *old, int new)
> -{
> -	int r, o = *old;
> -	r = arch_atomic_cmpxchg_release(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic_try_cmpxchg_release arch_atomic_try_cmpxchg_release
> -#endif
> -
> -#ifndef arch_atomic_try_cmpxchg_relaxed
> -static __always_inline bool
> -arch_atomic_try_cmpxchg_relaxed(atomic_t *v, int *old, int new)
> -{
> -	int r, o = *old;
> -	r = arch_atomic_cmpxchg_relaxed(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic_try_cmpxchg_relaxed arch_atomic_try_cmpxchg_relaxed
> -#endif
> -
> -#else /* arch_atomic_try_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic_try_cmpxchg_acquire
> -static __always_inline bool
> -arch_atomic_try_cmpxchg_acquire(atomic_t *v, int *old, int new)
> -{
> -	bool ret = arch_atomic_try_cmpxchg_relaxed(v, old, new);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic_try_cmpxchg_acquire arch_atomic_try_cmpxchg_acquire
> -#endif
> -
> -#ifndef arch_atomic_try_cmpxchg_release
> -static __always_inline bool
> -arch_atomic_try_cmpxchg_release(atomic_t *v, int *old, int new)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic_try_cmpxchg_relaxed(v, old, new);
> -}
> -#define arch_atomic_try_cmpxchg_release arch_atomic_try_cmpxchg_release
> -#endif
> -
> -#ifndef arch_atomic_try_cmpxchg
> -static __always_inline bool
> -arch_atomic_try_cmpxchg(atomic_t *v, int *old, int new)
> -{
> -	bool ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic_try_cmpxchg_relaxed(v, old, new);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic_try_cmpxchg arch_atomic_try_cmpxchg
> -#endif
> -
> -#endif /* arch_atomic_try_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic_sub_and_test
> -/**
> - * arch_atomic_sub_and_test - subtract value from variable and test result
> - * @i: integer value to subtract
> - * @v: pointer of type atomic_t
> - *
> - * Atomically subtracts @i from @v and returns
> - * true if the result is zero, or false for all
> - * other cases.
> - */
> -static __always_inline bool
> -arch_atomic_sub_and_test(int i, atomic_t *v)
> -{
> -	return arch_atomic_sub_return(i, v) == 0;
> -}
> -#define arch_atomic_sub_and_test arch_atomic_sub_and_test
> -#endif
> -
> -#ifndef arch_atomic_dec_and_test
> -/**
> - * arch_atomic_dec_and_test - decrement and test
> - * @v: pointer of type atomic_t
> - *
> - * Atomically decrements @v by 1 and
> - * returns true if the result is 0, or false for all other
> - * cases.
> - */
> -static __always_inline bool
> -arch_atomic_dec_and_test(atomic_t *v)
> -{
> -	return arch_atomic_dec_return(v) == 0;
> -}
> -#define arch_atomic_dec_and_test arch_atomic_dec_and_test
> -#endif
> -
> -#ifndef arch_atomic_inc_and_test
> -/**
> - * arch_atomic_inc_and_test - increment and test
> - * @v: pointer of type atomic_t
> - *
> - * Atomically increments @v by 1
> - * and returns true if the result is zero, or false for all
> - * other cases.
> - */
> -static __always_inline bool
> -arch_atomic_inc_and_test(atomic_t *v)
> -{
> -	return arch_atomic_inc_return(v) == 0;
> -}
> -#define arch_atomic_inc_and_test arch_atomic_inc_and_test
> -#endif
> -
> -#ifndef arch_atomic_add_negative
> -/**
> - * arch_atomic_add_negative - add and test if negative
> - * @i: integer value to add
> - * @v: pointer of type atomic_t
> - *
> - * Atomically adds @i to @v and returns true
> - * if the result is negative, or false when
> - * result is greater than or equal to zero.
> - */
> -static __always_inline bool
> -arch_atomic_add_negative(int i, atomic_t *v)
> -{
> -	return arch_atomic_add_return(i, v) < 0;
> -}
> -#define arch_atomic_add_negative arch_atomic_add_negative
> -#endif
> -
> -#ifndef arch_atomic_fetch_add_unless
> -/**
> - * arch_atomic_fetch_add_unless - add unless the number is already a given value
> - * @v: pointer of type atomic_t
> - * @a: the amount to add to v...
> - * @u: ...unless v is equal to u.
> - *
> - * Atomically adds @a to @v, so long as @v was not already @u.
> - * Returns original value of @v
> - */
> -static __always_inline int
> -arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
> -{
> -	int c = arch_atomic_read(v);
> -
> -	do {
> -		if (unlikely(c == u))
> -			break;
> -	} while (!arch_atomic_try_cmpxchg(v, &c, c + a));
> -
> -	return c;
> -}
> -#define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless
> -#endif
> -
> -#ifndef arch_atomic_add_unless
> -/**
> - * arch_atomic_add_unless - add unless the number is already a given value
> - * @v: pointer of type atomic_t
> - * @a: the amount to add to v...
> - * @u: ...unless v is equal to u.
> - *
> - * Atomically adds @a to @v, if @v was not already @u.
> - * Returns true if the addition was done.
> - */
> -static __always_inline bool
> -arch_atomic_add_unless(atomic_t *v, int a, int u)
> -{
> -	return arch_atomic_fetch_add_unless(v, a, u) != u;
> -}
> -#define arch_atomic_add_unless arch_atomic_add_unless
> -#endif
> -
> -#ifndef arch_atomic_inc_not_zero
> -/**
> - * arch_atomic_inc_not_zero - increment unless the number is zero
> - * @v: pointer of type atomic_t
> - *
> - * Atomically increments @v by 1, if @v is non-zero.
> - * Returns true if the increment was done.
> - */
> -static __always_inline bool
> -arch_atomic_inc_not_zero(atomic_t *v)
> -{
> -	return arch_atomic_add_unless(v, 1, 0);
> -}
> -#define arch_atomic_inc_not_zero arch_atomic_inc_not_zero
> -#endif
> -
> -#ifndef arch_atomic_inc_unless_negative
> -static __always_inline bool
> -arch_atomic_inc_unless_negative(atomic_t *v)
> -{
> -	int c = arch_atomic_read(v);
> -
> -	do {
> -		if (unlikely(c < 0))
> -			return false;
> -	} while (!arch_atomic_try_cmpxchg(v, &c, c + 1));
> -
> -	return true;
> -}
> -#define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative
> -#endif
> -
> -#ifndef arch_atomic_dec_unless_positive
> -static __always_inline bool
> -arch_atomic_dec_unless_positive(atomic_t *v)
> -{
> -	int c = arch_atomic_read(v);
> -
> -	do {
> -		if (unlikely(c > 0))
> -			return false;
> -	} while (!arch_atomic_try_cmpxchg(v, &c, c - 1));
> -
> -	return true;
> -}
> -#define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive
> -#endif
> -
> -#ifndef arch_atomic_dec_if_positive
> -static __always_inline int
> -arch_atomic_dec_if_positive(atomic_t *v)
> -{
> -	int dec, c = arch_atomic_read(v);
> -
> -	do {
> -		dec = c - 1;
> -		if (unlikely(dec < 0))
> -			break;
> -	} while (!arch_atomic_try_cmpxchg(v, &c, dec));
> -
> -	return dec;
> -}
> -#define arch_atomic_dec_if_positive arch_atomic_dec_if_positive
> -#endif
> -
> -#ifdef CONFIG_GENERIC_ATOMIC64
> -#include <asm-generic/atomic64.h>
> -#endif
> -
> -#ifndef arch_atomic64_read_acquire
> -static __always_inline s64
> -arch_atomic64_read_acquire(const atomic64_t *v)
> -{
> -	s64 ret;
> -
> -	if (__native_word(atomic64_t)) {
> -		ret = smp_load_acquire(&(v)->counter);
> -	} else {
> -		ret = arch_atomic64_read(v);
> -		__atomic_acquire_fence();
> -	}
> -
> -	return ret;
> -}
> -#define arch_atomic64_read_acquire arch_atomic64_read_acquire
> -#endif
> -
> -#ifndef arch_atomic64_set_release
> -static __always_inline void
> -arch_atomic64_set_release(atomic64_t *v, s64 i)
> -{
> -	if (__native_word(atomic64_t)) {
> -		smp_store_release(&(v)->counter, i);
> -	} else {
> -		__atomic_release_fence();
> -		arch_atomic64_set(v, i);
> -	}
> -}
> -#define arch_atomic64_set_release arch_atomic64_set_release
> -#endif
> -
> -#ifndef arch_atomic64_add_return_relaxed
> -#define arch_atomic64_add_return_acquire arch_atomic64_add_return
> -#define arch_atomic64_add_return_release arch_atomic64_add_return
> -#define arch_atomic64_add_return_relaxed arch_atomic64_add_return
> -#else /* arch_atomic64_add_return_relaxed */
> -
> -#ifndef arch_atomic64_add_return_acquire
> -static __always_inline s64
> -arch_atomic64_add_return_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_add_return_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_add_return_acquire arch_atomic64_add_return_acquire
> -#endif
> -
> -#ifndef arch_atomic64_add_return_release
> -static __always_inline s64
> -arch_atomic64_add_return_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_add_return_relaxed(i, v);
> -}
> -#define arch_atomic64_add_return_release arch_atomic64_add_return_release
> -#endif
> -
> -#ifndef arch_atomic64_add_return
> -static __always_inline s64
> -arch_atomic64_add_return(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_add_return_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_add_return arch_atomic64_add_return
> -#endif
> -
> -#endif /* arch_atomic64_add_return_relaxed */
> -
> -#ifndef arch_atomic64_fetch_add_relaxed
> -#define arch_atomic64_fetch_add_acquire arch_atomic64_fetch_add
> -#define arch_atomic64_fetch_add_release arch_atomic64_fetch_add
> -#define arch_atomic64_fetch_add_relaxed arch_atomic64_fetch_add
> -#else /* arch_atomic64_fetch_add_relaxed */
> -
> -#ifndef arch_atomic64_fetch_add_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_add_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_add_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_add_acquire arch_atomic64_fetch_add_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_add_release
> -static __always_inline s64
> -arch_atomic64_fetch_add_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_add_relaxed(i, v);
> -}
> -#define arch_atomic64_fetch_add_release arch_atomic64_fetch_add_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_add
> -static __always_inline s64
> -arch_atomic64_fetch_add(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_add_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_add arch_atomic64_fetch_add
> -#endif
> -
> -#endif /* arch_atomic64_fetch_add_relaxed */
> -
> -#ifndef arch_atomic64_sub_return_relaxed
> -#define arch_atomic64_sub_return_acquire arch_atomic64_sub_return
> -#define arch_atomic64_sub_return_release arch_atomic64_sub_return
> -#define arch_atomic64_sub_return_relaxed arch_atomic64_sub_return
> -#else /* arch_atomic64_sub_return_relaxed */
> -
> -#ifndef arch_atomic64_sub_return_acquire
> -static __always_inline s64
> -arch_atomic64_sub_return_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_sub_return_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_sub_return_acquire arch_atomic64_sub_return_acquire
> -#endif
> -
> -#ifndef arch_atomic64_sub_return_release
> -static __always_inline s64
> -arch_atomic64_sub_return_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_sub_return_relaxed(i, v);
> -}
> -#define arch_atomic64_sub_return_release arch_atomic64_sub_return_release
> -#endif
> -
> -#ifndef arch_atomic64_sub_return
> -static __always_inline s64
> -arch_atomic64_sub_return(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_sub_return_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_sub_return arch_atomic64_sub_return
> -#endif
> -
> -#endif /* arch_atomic64_sub_return_relaxed */
> -
> -#ifndef arch_atomic64_fetch_sub_relaxed
> -#define arch_atomic64_fetch_sub_acquire arch_atomic64_fetch_sub
> -#define arch_atomic64_fetch_sub_release arch_atomic64_fetch_sub
> -#define arch_atomic64_fetch_sub_relaxed arch_atomic64_fetch_sub
> -#else /* arch_atomic64_fetch_sub_relaxed */
> -
> -#ifndef arch_atomic64_fetch_sub_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_sub_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_sub_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_sub_acquire arch_atomic64_fetch_sub_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_sub_release
> -static __always_inline s64
> -arch_atomic64_fetch_sub_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_sub_relaxed(i, v);
> -}
> -#define arch_atomic64_fetch_sub_release arch_atomic64_fetch_sub_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_sub
> -static __always_inline s64
> -arch_atomic64_fetch_sub(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_sub_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_sub arch_atomic64_fetch_sub
> -#endif
> -
> -#endif /* arch_atomic64_fetch_sub_relaxed */
> -
> -#ifndef arch_atomic64_inc
> -static __always_inline void
> -arch_atomic64_inc(atomic64_t *v)
> -{
> -	arch_atomic64_add(1, v);
> -}
> -#define arch_atomic64_inc arch_atomic64_inc
> -#endif
> -
> -#ifndef arch_atomic64_inc_return_relaxed
> -#ifdef arch_atomic64_inc_return
> -#define arch_atomic64_inc_return_acquire arch_atomic64_inc_return
> -#define arch_atomic64_inc_return_release arch_atomic64_inc_return
> -#define arch_atomic64_inc_return_relaxed arch_atomic64_inc_return
> -#endif /* arch_atomic64_inc_return */
> -
> -#ifndef arch_atomic64_inc_return
> -static __always_inline s64
> -arch_atomic64_inc_return(atomic64_t *v)
> -{
> -	return arch_atomic64_add_return(1, v);
> -}
> -#define arch_atomic64_inc_return arch_atomic64_inc_return
> -#endif
> -
> -#ifndef arch_atomic64_inc_return_acquire
> -static __always_inline s64
> -arch_atomic64_inc_return_acquire(atomic64_t *v)
> -{
> -	return arch_atomic64_add_return_acquire(1, v);
> -}
> -#define arch_atomic64_inc_return_acquire arch_atomic64_inc_return_acquire
> -#endif
> -
> -#ifndef arch_atomic64_inc_return_release
> -static __always_inline s64
> -arch_atomic64_inc_return_release(atomic64_t *v)
> -{
> -	return arch_atomic64_add_return_release(1, v);
> -}
> -#define arch_atomic64_inc_return_release arch_atomic64_inc_return_release
> -#endif
> -
> -#ifndef arch_atomic64_inc_return_relaxed
> -static __always_inline s64
> -arch_atomic64_inc_return_relaxed(atomic64_t *v)
> -{
> -	return arch_atomic64_add_return_relaxed(1, v);
> -}
> -#define arch_atomic64_inc_return_relaxed arch_atomic64_inc_return_relaxed
> -#endif
> -
> -#else /* arch_atomic64_inc_return_relaxed */
> -
> -#ifndef arch_atomic64_inc_return_acquire
> -static __always_inline s64
> -arch_atomic64_inc_return_acquire(atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_inc_return_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_inc_return_acquire arch_atomic64_inc_return_acquire
> -#endif
> -
> -#ifndef arch_atomic64_inc_return_release
> -static __always_inline s64
> -arch_atomic64_inc_return_release(atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_inc_return_relaxed(v);
> -}
> -#define arch_atomic64_inc_return_release arch_atomic64_inc_return_release
> -#endif
> -
> -#ifndef arch_atomic64_inc_return
> -static __always_inline s64
> -arch_atomic64_inc_return(atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_inc_return_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_inc_return arch_atomic64_inc_return
> -#endif
> -
> -#endif /* arch_atomic64_inc_return_relaxed */
> -
> -#ifndef arch_atomic64_fetch_inc_relaxed
> -#ifdef arch_atomic64_fetch_inc
> -#define arch_atomic64_fetch_inc_acquire arch_atomic64_fetch_inc
> -#define arch_atomic64_fetch_inc_release arch_atomic64_fetch_inc
> -#define arch_atomic64_fetch_inc_relaxed arch_atomic64_fetch_inc
> -#endif /* arch_atomic64_fetch_inc */
> -
> -#ifndef arch_atomic64_fetch_inc
> -static __always_inline s64
> -arch_atomic64_fetch_inc(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_add(1, v);
> -}
> -#define arch_atomic64_fetch_inc arch_atomic64_fetch_inc
> -#endif
> -
> -#ifndef arch_atomic64_fetch_inc_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_inc_acquire(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_add_acquire(1, v);
> -}
> -#define arch_atomic64_fetch_inc_acquire arch_atomic64_fetch_inc_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_inc_release
> -static __always_inline s64
> -arch_atomic64_fetch_inc_release(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_add_release(1, v);
> -}
> -#define arch_atomic64_fetch_inc_release arch_atomic64_fetch_inc_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_inc_relaxed
> -static __always_inline s64
> -arch_atomic64_fetch_inc_relaxed(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_add_relaxed(1, v);
> -}
> -#define arch_atomic64_fetch_inc_relaxed arch_atomic64_fetch_inc_relaxed
> -#endif
> -
> -#else /* arch_atomic64_fetch_inc_relaxed */
> -
> -#ifndef arch_atomic64_fetch_inc_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_inc_acquire(atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_inc_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_inc_acquire arch_atomic64_fetch_inc_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_inc_release
> -static __always_inline s64
> -arch_atomic64_fetch_inc_release(atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_inc_relaxed(v);
> -}
> -#define arch_atomic64_fetch_inc_release arch_atomic64_fetch_inc_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_inc
> -static __always_inline s64
> -arch_atomic64_fetch_inc(atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_inc_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_inc arch_atomic64_fetch_inc
> -#endif
> -
> -#endif /* arch_atomic64_fetch_inc_relaxed */
> -
> -#ifndef arch_atomic64_dec
> -static __always_inline void
> -arch_atomic64_dec(atomic64_t *v)
> -{
> -	arch_atomic64_sub(1, v);
> -}
> -#define arch_atomic64_dec arch_atomic64_dec
> -#endif
> -
> -#ifndef arch_atomic64_dec_return_relaxed
> -#ifdef arch_atomic64_dec_return
> -#define arch_atomic64_dec_return_acquire arch_atomic64_dec_return
> -#define arch_atomic64_dec_return_release arch_atomic64_dec_return
> -#define arch_atomic64_dec_return_relaxed arch_atomic64_dec_return
> -#endif /* arch_atomic64_dec_return */
> -
> -#ifndef arch_atomic64_dec_return
> -static __always_inline s64
> -arch_atomic64_dec_return(atomic64_t *v)
> -{
> -	return arch_atomic64_sub_return(1, v);
> -}
> -#define arch_atomic64_dec_return arch_atomic64_dec_return
> -#endif
> -
> -#ifndef arch_atomic64_dec_return_acquire
> -static __always_inline s64
> -arch_atomic64_dec_return_acquire(atomic64_t *v)
> -{
> -	return arch_atomic64_sub_return_acquire(1, v);
> -}
> -#define arch_atomic64_dec_return_acquire arch_atomic64_dec_return_acquire
> -#endif
> -
> -#ifndef arch_atomic64_dec_return_release
> -static __always_inline s64
> -arch_atomic64_dec_return_release(atomic64_t *v)
> -{
> -	return arch_atomic64_sub_return_release(1, v);
> -}
> -#define arch_atomic64_dec_return_release arch_atomic64_dec_return_release
> -#endif
> -
> -#ifndef arch_atomic64_dec_return_relaxed
> -static __always_inline s64
> -arch_atomic64_dec_return_relaxed(atomic64_t *v)
> -{
> -	return arch_atomic64_sub_return_relaxed(1, v);
> -}
> -#define arch_atomic64_dec_return_relaxed arch_atomic64_dec_return_relaxed
> -#endif
> -
> -#else /* arch_atomic64_dec_return_relaxed */
> -
> -#ifndef arch_atomic64_dec_return_acquire
> -static __always_inline s64
> -arch_atomic64_dec_return_acquire(atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_dec_return_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_dec_return_acquire arch_atomic64_dec_return_acquire
> -#endif
> -
> -#ifndef arch_atomic64_dec_return_release
> -static __always_inline s64
> -arch_atomic64_dec_return_release(atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_dec_return_relaxed(v);
> -}
> -#define arch_atomic64_dec_return_release arch_atomic64_dec_return_release
> -#endif
> -
> -#ifndef arch_atomic64_dec_return
> -static __always_inline s64
> -arch_atomic64_dec_return(atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_dec_return_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_dec_return arch_atomic64_dec_return
> -#endif
> -
> -#endif /* arch_atomic64_dec_return_relaxed */
> -
> -#ifndef arch_atomic64_fetch_dec_relaxed
> -#ifdef arch_atomic64_fetch_dec
> -#define arch_atomic64_fetch_dec_acquire arch_atomic64_fetch_dec
> -#define arch_atomic64_fetch_dec_release arch_atomic64_fetch_dec
> -#define arch_atomic64_fetch_dec_relaxed arch_atomic64_fetch_dec
> -#endif /* arch_atomic64_fetch_dec */
> -
> -#ifndef arch_atomic64_fetch_dec
> -static __always_inline s64
> -arch_atomic64_fetch_dec(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_sub(1, v);
> -}
> -#define arch_atomic64_fetch_dec arch_atomic64_fetch_dec
> -#endif
> -
> -#ifndef arch_atomic64_fetch_dec_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_dec_acquire(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_sub_acquire(1, v);
> -}
> -#define arch_atomic64_fetch_dec_acquire arch_atomic64_fetch_dec_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_dec_release
> -static __always_inline s64
> -arch_atomic64_fetch_dec_release(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_sub_release(1, v);
> -}
> -#define arch_atomic64_fetch_dec_release arch_atomic64_fetch_dec_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_dec_relaxed
> -static __always_inline s64
> -arch_atomic64_fetch_dec_relaxed(atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_sub_relaxed(1, v);
> -}
> -#define arch_atomic64_fetch_dec_relaxed arch_atomic64_fetch_dec_relaxed
> -#endif
> -
> -#else /* arch_atomic64_fetch_dec_relaxed */
> -
> -#ifndef arch_atomic64_fetch_dec_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_dec_acquire(atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_dec_relaxed(v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_dec_acquire arch_atomic64_fetch_dec_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_dec_release
> -static __always_inline s64
> -arch_atomic64_fetch_dec_release(atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_dec_relaxed(v);
> -}
> -#define arch_atomic64_fetch_dec_release arch_atomic64_fetch_dec_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_dec
> -static __always_inline s64
> -arch_atomic64_fetch_dec(atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_dec_relaxed(v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_dec arch_atomic64_fetch_dec
> -#endif
> -
> -#endif /* arch_atomic64_fetch_dec_relaxed */
> -
> -#ifndef arch_atomic64_fetch_and_relaxed
> -#define arch_atomic64_fetch_and_acquire arch_atomic64_fetch_and
> -#define arch_atomic64_fetch_and_release arch_atomic64_fetch_and
> -#define arch_atomic64_fetch_and_relaxed arch_atomic64_fetch_and
> -#else /* arch_atomic64_fetch_and_relaxed */
> -
> -#ifndef arch_atomic64_fetch_and_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_and_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_and_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_and_acquire arch_atomic64_fetch_and_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_and_release
> -static __always_inline s64
> -arch_atomic64_fetch_and_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_and_relaxed(i, v);
> -}
> -#define arch_atomic64_fetch_and_release arch_atomic64_fetch_and_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_and
> -static __always_inline s64
> -arch_atomic64_fetch_and(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_and_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_and arch_atomic64_fetch_and
> -#endif
> -
> -#endif /* arch_atomic64_fetch_and_relaxed */
> -
> -#ifndef arch_atomic64_andnot
> -static __always_inline void
> -arch_atomic64_andnot(s64 i, atomic64_t *v)
> -{
> -	arch_atomic64_and(~i, v);
> -}
> -#define arch_atomic64_andnot arch_atomic64_andnot
> -#endif
> -
> -#ifndef arch_atomic64_fetch_andnot_relaxed
> -#ifdef arch_atomic64_fetch_andnot
> -#define arch_atomic64_fetch_andnot_acquire arch_atomic64_fetch_andnot
> -#define arch_atomic64_fetch_andnot_release arch_atomic64_fetch_andnot
> -#define arch_atomic64_fetch_andnot_relaxed arch_atomic64_fetch_andnot
> -#endif /* arch_atomic64_fetch_andnot */
> -
> -#ifndef arch_atomic64_fetch_andnot
> -static __always_inline s64
> -arch_atomic64_fetch_andnot(s64 i, atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_and(~i, v);
> -}
> -#define arch_atomic64_fetch_andnot arch_atomic64_fetch_andnot
> -#endif
> -
> -#ifndef arch_atomic64_fetch_andnot_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_andnot_acquire(s64 i, atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_and_acquire(~i, v);
> -}
> -#define arch_atomic64_fetch_andnot_acquire arch_atomic64_fetch_andnot_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_andnot_release
> -static __always_inline s64
> -arch_atomic64_fetch_andnot_release(s64 i, atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_and_release(~i, v);
> -}
> -#define arch_atomic64_fetch_andnot_release arch_atomic64_fetch_andnot_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_andnot_relaxed
> -static __always_inline s64
> -arch_atomic64_fetch_andnot_relaxed(s64 i, atomic64_t *v)
> -{
> -	return arch_atomic64_fetch_and_relaxed(~i, v);
> -}
> -#define arch_atomic64_fetch_andnot_relaxed arch_atomic64_fetch_andnot_relaxed
> -#endif
> -
> -#else /* arch_atomic64_fetch_andnot_relaxed */
> -
> -#ifndef arch_atomic64_fetch_andnot_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_andnot_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_andnot_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_andnot_acquire arch_atomic64_fetch_andnot_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_andnot_release
> -static __always_inline s64
> -arch_atomic64_fetch_andnot_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_andnot_relaxed(i, v);
> -}
> -#define arch_atomic64_fetch_andnot_release arch_atomic64_fetch_andnot_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_andnot
> -static __always_inline s64
> -arch_atomic64_fetch_andnot(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_andnot_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_andnot arch_atomic64_fetch_andnot
> -#endif
> -
> -#endif /* arch_atomic64_fetch_andnot_relaxed */
> -
> -#ifndef arch_atomic64_fetch_or_relaxed
> -#define arch_atomic64_fetch_or_acquire arch_atomic64_fetch_or
> -#define arch_atomic64_fetch_or_release arch_atomic64_fetch_or
> -#define arch_atomic64_fetch_or_relaxed arch_atomic64_fetch_or
> -#else /* arch_atomic64_fetch_or_relaxed */
> -
> -#ifndef arch_atomic64_fetch_or_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_or_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_or_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_or_acquire arch_atomic64_fetch_or_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_or_release
> -static __always_inline s64
> -arch_atomic64_fetch_or_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_or_relaxed(i, v);
> -}
> -#define arch_atomic64_fetch_or_release arch_atomic64_fetch_or_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_or
> -static __always_inline s64
> -arch_atomic64_fetch_or(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_or_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_or arch_atomic64_fetch_or
> -#endif
> -
> -#endif /* arch_atomic64_fetch_or_relaxed */
> -
> -#ifndef arch_atomic64_fetch_xor_relaxed
> -#define arch_atomic64_fetch_xor_acquire arch_atomic64_fetch_xor
> -#define arch_atomic64_fetch_xor_release arch_atomic64_fetch_xor
> -#define arch_atomic64_fetch_xor_relaxed arch_atomic64_fetch_xor
> -#else /* arch_atomic64_fetch_xor_relaxed */
> -
> -#ifndef arch_atomic64_fetch_xor_acquire
> -static __always_inline s64
> -arch_atomic64_fetch_xor_acquire(s64 i, atomic64_t *v)
> -{
> -	s64 ret = arch_atomic64_fetch_xor_relaxed(i, v);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_xor_acquire arch_atomic64_fetch_xor_acquire
> -#endif
> -
> -#ifndef arch_atomic64_fetch_xor_release
> -static __always_inline s64
> -arch_atomic64_fetch_xor_release(s64 i, atomic64_t *v)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_fetch_xor_relaxed(i, v);
> -}
> -#define arch_atomic64_fetch_xor_release arch_atomic64_fetch_xor_release
> -#endif
> -
> -#ifndef arch_atomic64_fetch_xor
> -static __always_inline s64
> -arch_atomic64_fetch_xor(s64 i, atomic64_t *v)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_fetch_xor_relaxed(i, v);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_fetch_xor arch_atomic64_fetch_xor
> -#endif
> -
> -#endif /* arch_atomic64_fetch_xor_relaxed */
> -
> -#ifndef arch_atomic64_xchg_relaxed
> -#define arch_atomic64_xchg_acquire arch_atomic64_xchg
> -#define arch_atomic64_xchg_release arch_atomic64_xchg
> -#define arch_atomic64_xchg_relaxed arch_atomic64_xchg
> -#else /* arch_atomic64_xchg_relaxed */
> -
> -#ifndef arch_atomic64_xchg_acquire
> -static __always_inline s64
> -arch_atomic64_xchg_acquire(atomic64_t *v, s64 i)
> -{
> -	s64 ret = arch_atomic64_xchg_relaxed(v, i);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_xchg_acquire arch_atomic64_xchg_acquire
> -#endif
> -
> -#ifndef arch_atomic64_xchg_release
> -static __always_inline s64
> -arch_atomic64_xchg_release(atomic64_t *v, s64 i)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_xchg_relaxed(v, i);
> -}
> -#define arch_atomic64_xchg_release arch_atomic64_xchg_release
> -#endif
> -
> -#ifndef arch_atomic64_xchg
> -static __always_inline s64
> -arch_atomic64_xchg(atomic64_t *v, s64 i)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_xchg_relaxed(v, i);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_xchg arch_atomic64_xchg
> -#endif
> -
> -#endif /* arch_atomic64_xchg_relaxed */
> -
> -#ifndef arch_atomic64_cmpxchg_relaxed
> -#define arch_atomic64_cmpxchg_acquire arch_atomic64_cmpxchg
> -#define arch_atomic64_cmpxchg_release arch_atomic64_cmpxchg
> -#define arch_atomic64_cmpxchg_relaxed arch_atomic64_cmpxchg
> -#else /* arch_atomic64_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic64_cmpxchg_acquire
> -static __always_inline s64
> -arch_atomic64_cmpxchg_acquire(atomic64_t *v, s64 old, s64 new)
> -{
> -	s64 ret = arch_atomic64_cmpxchg_relaxed(v, old, new);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_cmpxchg_acquire arch_atomic64_cmpxchg_acquire
> -#endif
> -
> -#ifndef arch_atomic64_cmpxchg_release
> -static __always_inline s64
> -arch_atomic64_cmpxchg_release(atomic64_t *v, s64 old, s64 new)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_cmpxchg_relaxed(v, old, new);
> -}
> -#define arch_atomic64_cmpxchg_release arch_atomic64_cmpxchg_release
> -#endif
> -
> -#ifndef arch_atomic64_cmpxchg
> -static __always_inline s64
> -arch_atomic64_cmpxchg(atomic64_t *v, s64 old, s64 new)
> -{
> -	s64 ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_cmpxchg_relaxed(v, old, new);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_cmpxchg arch_atomic64_cmpxchg
> -#endif
> -
> -#endif /* arch_atomic64_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic64_try_cmpxchg_relaxed
> -#ifdef arch_atomic64_try_cmpxchg
> -#define arch_atomic64_try_cmpxchg_acquire arch_atomic64_try_cmpxchg
> -#define arch_atomic64_try_cmpxchg_release arch_atomic64_try_cmpxchg
> -#define arch_atomic64_try_cmpxchg_relaxed arch_atomic64_try_cmpxchg
> -#endif /* arch_atomic64_try_cmpxchg */
> -
> -#ifndef arch_atomic64_try_cmpxchg
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg(atomic64_t *v, s64 *old, s64 new)
> -{
> -	s64 r, o = *old;
> -	r = arch_atomic64_cmpxchg(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic64_try_cmpxchg arch_atomic64_try_cmpxchg
> -#endif
> -
> -#ifndef arch_atomic64_try_cmpxchg_acquire
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg_acquire(atomic64_t *v, s64 *old, s64 new)
> -{
> -	s64 r, o = *old;
> -	r = arch_atomic64_cmpxchg_acquire(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic64_try_cmpxchg_acquire arch_atomic64_try_cmpxchg_acquire
> -#endif
> -
> -#ifndef arch_atomic64_try_cmpxchg_release
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg_release(atomic64_t *v, s64 *old, s64 new)
> -{
> -	s64 r, o = *old;
> -	r = arch_atomic64_cmpxchg_release(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic64_try_cmpxchg_release arch_atomic64_try_cmpxchg_release
> -#endif
> -
> -#ifndef arch_atomic64_try_cmpxchg_relaxed
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg_relaxed(atomic64_t *v, s64 *old, s64 new)
> -{
> -	s64 r, o = *old;
> -	r = arch_atomic64_cmpxchg_relaxed(v, o, new);
> -	if (unlikely(r != o))
> -		*old = r;
> -	return likely(r == o);
> -}
> -#define arch_atomic64_try_cmpxchg_relaxed arch_atomic64_try_cmpxchg_relaxed
> -#endif
> -
> -#else /* arch_atomic64_try_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic64_try_cmpxchg_acquire
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg_acquire(atomic64_t *v, s64 *old, s64 new)
> -{
> -	bool ret = arch_atomic64_try_cmpxchg_relaxed(v, old, new);
> -	__atomic_acquire_fence();
> -	return ret;
> -}
> -#define arch_atomic64_try_cmpxchg_acquire arch_atomic64_try_cmpxchg_acquire
> -#endif
> -
> -#ifndef arch_atomic64_try_cmpxchg_release
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg_release(atomic64_t *v, s64 *old, s64 new)
> -{
> -	__atomic_release_fence();
> -	return arch_atomic64_try_cmpxchg_relaxed(v, old, new);
> -}
> -#define arch_atomic64_try_cmpxchg_release arch_atomic64_try_cmpxchg_release
> -#endif
> -
> -#ifndef arch_atomic64_try_cmpxchg
> -static __always_inline bool
> -arch_atomic64_try_cmpxchg(atomic64_t *v, s64 *old, s64 new)
> -{
> -	bool ret;
> -	__atomic_pre_full_fence();
> -	ret = arch_atomic64_try_cmpxchg_relaxed(v, old, new);
> -	__atomic_post_full_fence();
> -	return ret;
> -}
> -#define arch_atomic64_try_cmpxchg arch_atomic64_try_cmpxchg
> -#endif
> -
> -#endif /* arch_atomic64_try_cmpxchg_relaxed */
> -
> -#ifndef arch_atomic64_sub_and_test
> -/**
> - * arch_atomic64_sub_and_test - subtract value from variable and test result
> - * @i: integer value to subtract
> - * @v: pointer of type atomic64_t
> - *
> - * Atomically subtracts @i from @v and returns
> - * true if the result is zero, or false for all
> - * other cases.
> - */
> -static __always_inline bool
> -arch_atomic64_sub_and_test(s64 i, atomic64_t *v)
> -{
> -	return arch_atomic64_sub_return(i, v) == 0;
> -}
> -#define arch_atomic64_sub_and_test arch_atomic64_sub_and_test
> -#endif
> -
> -#ifndef arch_atomic64_dec_and_test
> -/**
> - * arch_atomic64_dec_and_test - decrement and test
> - * @v: pointer of type atomic64_t
> - *
> - * Atomically decrements @v by 1 and
> - * returns true if the result is 0, or false for all other
> - * cases.
> - */
> -static __always_inline bool
> -arch_atomic64_dec_and_test(atomic64_t *v)
> -{
> -	return arch_atomic64_dec_return(v) == 0;
> -}
> -#define arch_atomic64_dec_and_test arch_atomic64_dec_and_test
> -#endif
> -
> -#ifndef arch_atomic64_inc_and_test
> -/**
> - * arch_atomic64_inc_and_test - increment and test
> - * @v: pointer of type atomic64_t
> - *
> - * Atomically increments @v by 1
> - * and returns true if the result is zero, or false for all
> - * other cases.
> - */
> -static __always_inline bool
> -arch_atomic64_inc_and_test(atomic64_t *v)
> -{
> -	return arch_atomic64_inc_return(v) == 0;
> -}
> -#define arch_atomic64_inc_and_test arch_atomic64_inc_and_test
> -#endif
> -
> -#ifndef arch_atomic64_add_negative
> -/**
> - * arch_atomic64_add_negative - add and test if negative
> - * @i: integer value to add
> - * @v: pointer of type atomic64_t
> - *
> - * Atomically adds @i to @v and returns true
> - * if the result is negative, or false when
> - * result is greater than or equal to zero.
> - */
> -static __always_inline bool
> -arch_atomic64_add_negative(s64 i, atomic64_t *v)
> -{
> -	return arch_atomic64_add_return(i, v) < 0;
> -}
> -#define arch_atomic64_add_negative arch_atomic64_add_negative
> -#endif
> -
> -#ifndef arch_atomic64_fetch_add_unless
> -/**
> - * arch_atomic64_fetch_add_unless - add unless the number is already a given value
> - * @v: pointer of type atomic64_t
> - * @a: the amount to add to v...
> - * @u: ...unless v is equal to u.
> - *
> - * Atomically adds @a to @v, so long as @v was not already @u.
> - * Returns original value of @v
> - */
> -static __always_inline s64
> -arch_atomic64_fetch_add_unless(atomic64_t *v, s64 a, s64 u)
> -{
> -	s64 c = arch_atomic64_read(v);
> -
> -	do {
> -		if (unlikely(c == u))
> -			break;
> -	} while (!arch_atomic64_try_cmpxchg(v, &c, c + a));
> -
> -	return c;
> -}
> -#define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless
> -#endif
> -
> -#ifndef arch_atomic64_add_unless
> -/**
> - * arch_atomic64_add_unless - add unless the number is already a given value
> - * @v: pointer of type atomic64_t
> - * @a: the amount to add to v...
> - * @u: ...unless v is equal to u.
> - *
> - * Atomically adds @a to @v, if @v was not already @u.
> - * Returns true if the addition was done.
> - */
> -static __always_inline bool
> -arch_atomic64_add_unless(atomic64_t *v, s64 a, s64 u)
> -{
> -	return arch_atomic64_fetch_add_unless(v, a, u) != u;
> -}
> -#define arch_atomic64_add_unless arch_atomic64_add_unless
> -#endif
> -
> -#ifndef arch_atomic64_inc_not_zero
> -/**
> - * arch_atomic64_inc_not_zero - increment unless the number is zero
> - * @v: pointer of type atomic64_t
> - *
> - * Atomically increments @v by 1, if @v is non-zero.
> - * Returns true if the increment was done.
> - */
> -static __always_inline bool
> -arch_atomic64_inc_not_zero(atomic64_t *v)
> -{
> -	return arch_atomic64_add_unless(v, 1, 0);
> -}
> -#define arch_atomic64_inc_not_zero arch_atomic64_inc_not_zero
> -#endif
> -
> -#ifndef arch_atomic64_inc_unless_negative
> -static __always_inline bool
> -arch_atomic64_inc_unless_negative(atomic64_t *v)
> -{
> -	s64 c = arch_atomic64_read(v);
> -
> -	do {
> -		if (unlikely(c < 0))
> -			return false;
> -	} while (!arch_atomic64_try_cmpxchg(v, &c, c + 1));
> -
> -	return true;
> -}
> -#define arch_atomic64_inc_unless_negative arch_atomic64_inc_unless_negative
> -#endif
> -
> -#ifndef arch_atomic64_dec_unless_positive
> -static __always_inline bool
> -arch_atomic64_dec_unless_positive(atomic64_t *v)
> -{
> -	s64 c = arch_atomic64_read(v);
> -
> -	do {
> -		if (unlikely(c > 0))
> -			return false;
> -	} while (!arch_atomic64_try_cmpxchg(v, &c, c - 1));
> -
> -	return true;
> -}
> -#define arch_atomic64_dec_unless_positive arch_atomic64_dec_unless_positive
> -#endif
> -
> -#ifndef arch_atomic64_dec_if_positive
> -static __always_inline s64
> -arch_atomic64_dec_if_positive(atomic64_t *v)
> -{
> -	s64 dec, c = arch_atomic64_read(v);
> -
> -	do {
> -		dec = c - 1;
> -		if (unlikely(dec < 0))
> -			break;
> -	} while (!arch_atomic64_try_cmpxchg(v, &c, dec));
> -
> -	return dec;
> -}
> -#define arch_atomic64_dec_if_positive arch_atomic64_dec_if_positive
> -#endif
> -
> -#endif /* _LINUX_ATOMIC_FALLBACK_H */
> -// 8e2cc06bc0d2c0967d2f8424762bd48555ee40ae
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#include <generated/atomic-arch-fallback.h>
> diff --git a/include/linux/atomic/atomic-instrumented.h b/include/linux/atomic/atomic-instrumented.h
> index 5d69b143c28e..4751a8e6bc13 100644
> --- a/include/linux/atomic/atomic-instrumented.h
> +++ b/include/linux/atomic/atomic-instrumented.h
> @@ -1,2048 +1,2 @@
> -// SPDX-License-Identifier: GPL-2.0
> -
> -// Generated by scripts/atomic/gen-atomic-instrumented.sh
> -// DO NOT MODIFY THIS FILE DIRECTLY
> -
> -/*
> - * This file provides wrappers with KASAN instrumentation for atomic operations.
> - * To use this functionality an arch's atomic.h file needs to define all
> - * atomic operations with arch_ prefix (e.g. arch_atomic_read()) and include
> - * this file at the end. This file provides atomic_read() that forwards to
> - * arch_atomic_read() for actual atomic operation.
> - * Note: if an arch atomic operation is implemented by means of other atomic
> - * operations (e.g. atomic_read()/atomic_cmpxchg() loop), then it needs to use
> - * arch_ variants (i.e. arch_atomic_read()/arch_atomic_cmpxchg()) to avoid
> - * double instrumentation.
> - */
> -#ifndef _LINUX_ATOMIC_INSTRUMENTED_H
> -#define _LINUX_ATOMIC_INSTRUMENTED_H
> -
> -#include <linux/build_bug.h>
> -#include <linux/compiler.h>
> -#include <linux/instrumented.h>
> -
> -static __always_inline int
> -atomic_read(const atomic_t *v)
> -{
> -	instrument_atomic_read(v, sizeof(*v));
> -	return arch_atomic_read(v);
> -}
> -
> -static __always_inline int
> -atomic_read_acquire(const atomic_t *v)
> -{
> -	instrument_atomic_read(v, sizeof(*v));
> -	return arch_atomic_read_acquire(v);
> -}
> -
> -static __always_inline void
> -atomic_set(atomic_t *v, int i)
> -{
> -	instrument_atomic_write(v, sizeof(*v));
> -	arch_atomic_set(v, i);
> -}
> -
> -static __always_inline void
> -atomic_set_release(atomic_t *v, int i)
> -{
> -	kcsan_release();
> -	instrument_atomic_write(v, sizeof(*v));
> -	arch_atomic_set_release(v, i);
> -}
> -
> -static __always_inline void
> -atomic_add(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_add(i, v);
> -}
> -
> -static __always_inline int
> -atomic_add_return(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_add_return(i, v);
> -}
> -
> -static __always_inline int
> -atomic_add_return_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_add_return_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_add_return_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_add_return_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_add_return_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_add_return_relaxed(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_add(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_add(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_add_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_add_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_add_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_add_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_add_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_add_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_sub(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_sub(i, v);
> -}
> -
> -static __always_inline int
> -atomic_sub_return(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_sub_return(i, v);
> -}
> -
> -static __always_inline int
> -atomic_sub_return_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_sub_return_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_sub_return_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_sub_return_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_sub_return_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_sub_return_relaxed(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_sub(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_sub(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_sub_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_sub_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_sub_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_sub_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_sub_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_sub_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_inc(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_inc(v);
> -}
> -
> -static __always_inline int
> -atomic_inc_return(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_return(v);
> -}
> -
> -static __always_inline int
> -atomic_inc_return_acquire(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_return_acquire(v);
> -}
> -
> -static __always_inline int
> -atomic_inc_return_release(atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_return_release(v);
> -}
> -
> -static __always_inline int
> -atomic_inc_return_relaxed(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_return_relaxed(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_inc(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_inc(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_inc_acquire(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_inc_acquire(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_inc_release(atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_inc_release(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_inc_relaxed(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_inc_relaxed(v);
> -}
> -
> -static __always_inline void
> -atomic_dec(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_dec(v);
> -}
> -
> -static __always_inline int
> -atomic_dec_return(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_return(v);
> -}
> -
> -static __always_inline int
> -atomic_dec_return_acquire(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_return_acquire(v);
> -}
> -
> -static __always_inline int
> -atomic_dec_return_release(atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_return_release(v);
> -}
> -
> -static __always_inline int
> -atomic_dec_return_relaxed(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_return_relaxed(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_dec(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_dec(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_dec_acquire(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_dec_acquire(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_dec_release(atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_dec_release(v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_dec_relaxed(atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_dec_relaxed(v);
> -}
> -
> -static __always_inline void
> -atomic_and(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_and(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_and(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_and(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_and_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_and_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_and_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_and_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_and_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_and_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_andnot(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_andnot(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_andnot(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_andnot(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_andnot_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_andnot_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_andnot_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_andnot_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_andnot_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_andnot_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_or(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_or(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_or(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_or(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_or_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_or_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_or_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_or_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_or_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_or_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_xor(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_xor(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_xor(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_xor(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_xor_acquire(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_xor_acquire(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_xor_release(int i, atomic_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_xor_release(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_xor_relaxed(int i, atomic_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_xor_relaxed(i, v);
> -}
> -
> -static __always_inline int
> -atomic_xchg(atomic_t *v, int i)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_xchg(v, i);
> -}
> -
> -static __always_inline int
> -atomic_xchg_acquire(atomic_t *v, int i)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_xchg_acquire(v, i);
> -}
> -
> -static __always_inline int
> -atomic_xchg_release(atomic_t *v, int i)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_xchg_release(v, i);
> -}
> -
> -static __always_inline int
> -atomic_xchg_relaxed(atomic_t *v, int i)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_xchg_relaxed(v, i);
> -}
> -
> -static __always_inline int
> -atomic_cmpxchg(atomic_t *v, int old, int new)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline int
> -atomic_cmpxchg_acquire(atomic_t *v, int old, int new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline int
> -atomic_cmpxchg_release(atomic_t *v, int old, int new)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline int
> -atomic_cmpxchg_relaxed(atomic_t *v, int old, int new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_try_cmpxchg(atomic_t *v, int *old, int new)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_try_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_try_cmpxchg_acquire(atomic_t *v, int *old, int new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_try_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_try_cmpxchg_release(atomic_t *v, int *old, int new)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_try_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_try_cmpxchg_relaxed(atomic_t *v, int *old, int new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_try_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_sub_and_test(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_sub_and_test(i, v);
> -}
> -
> -static __always_inline bool
> -atomic_dec_and_test(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_and_test(v);
> -}
> -
> -static __always_inline bool
> -atomic_inc_and_test(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_and_test(v);
> -}
> -
> -static __always_inline bool
> -atomic_add_negative(int i, atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_add_negative(i, v);
> -}
> -
> -static __always_inline int
> -atomic_fetch_add_unless(atomic_t *v, int a, int u)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_fetch_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -atomic_add_unless(atomic_t *v, int a, int u)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -atomic_inc_not_zero(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_not_zero(v);
> -}
> -
> -static __always_inline bool
> -atomic_inc_unless_negative(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_inc_unless_negative(v);
> -}
> -
> -static __always_inline bool
> -atomic_dec_unless_positive(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_unless_positive(v);
> -}
> -
> -static __always_inline int
> -atomic_dec_if_positive(atomic_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_dec_if_positive(v);
> -}
> -
> -static __always_inline s64
> -atomic64_read(const atomic64_t *v)
> -{
> -	instrument_atomic_read(v, sizeof(*v));
> -	return arch_atomic64_read(v);
> -}
> -
> -static __always_inline s64
> -atomic64_read_acquire(const atomic64_t *v)
> -{
> -	instrument_atomic_read(v, sizeof(*v));
> -	return arch_atomic64_read_acquire(v);
> -}
> -
> -static __always_inline void
> -atomic64_set(atomic64_t *v, s64 i)
> -{
> -	instrument_atomic_write(v, sizeof(*v));
> -	arch_atomic64_set(v, i);
> -}
> -
> -static __always_inline void
> -atomic64_set_release(atomic64_t *v, s64 i)
> -{
> -	kcsan_release();
> -	instrument_atomic_write(v, sizeof(*v));
> -	arch_atomic64_set_release(v, i);
> -}
> -
> -static __always_inline void
> -atomic64_add(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_add(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_add_return(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_add_return(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_add_return_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_add_return_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_add_return_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_add_return_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_add_return_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_add_return_relaxed(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_add(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_add(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_add_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_add_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_add_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_add_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_add_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_add_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic64_sub(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_sub(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_sub_return(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_sub_return(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_sub_return_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_sub_return_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_sub_return_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_sub_return_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_sub_return_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_sub_return_relaxed(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_sub(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_sub(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_sub_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_sub_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_sub_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_sub_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_sub_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_sub_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic64_inc(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_inc(v);
> -}
> -
> -static __always_inline s64
> -atomic64_inc_return(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_return(v);
> -}
> -
> -static __always_inline s64
> -atomic64_inc_return_acquire(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_return_acquire(v);
> -}
> -
> -static __always_inline s64
> -atomic64_inc_return_release(atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_return_release(v);
> -}
> -
> -static __always_inline s64
> -atomic64_inc_return_relaxed(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_return_relaxed(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_inc(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_inc(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_inc_acquire(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_inc_acquire(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_inc_release(atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_inc_release(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_inc_relaxed(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_inc_relaxed(v);
> -}
> -
> -static __always_inline void
> -atomic64_dec(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_dec(v);
> -}
> -
> -static __always_inline s64
> -atomic64_dec_return(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_return(v);
> -}
> -
> -static __always_inline s64
> -atomic64_dec_return_acquire(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_return_acquire(v);
> -}
> -
> -static __always_inline s64
> -atomic64_dec_return_release(atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_return_release(v);
> -}
> -
> -static __always_inline s64
> -atomic64_dec_return_relaxed(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_return_relaxed(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_dec(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_dec(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_dec_acquire(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_dec_acquire(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_dec_release(atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_dec_release(v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_dec_relaxed(atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_dec_relaxed(v);
> -}
> -
> -static __always_inline void
> -atomic64_and(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_and(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_and(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_and(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_and_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_and_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_and_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_and_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_and_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_and_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic64_andnot(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_andnot(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_andnot(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_andnot(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_andnot_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_andnot_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_andnot_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_andnot_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_andnot_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_andnot_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic64_or(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_or(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_or(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_or(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_or_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_or_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_or_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_or_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_or_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_or_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic64_xor(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic64_xor(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_xor(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_xor(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_xor_acquire(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_xor_acquire(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_xor_release(s64 i, atomic64_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_xor_release(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_xor_relaxed(s64 i, atomic64_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_xor_relaxed(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_xchg(atomic64_t *v, s64 i)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_xchg(v, i);
> -}
> -
> -static __always_inline s64
> -atomic64_xchg_acquire(atomic64_t *v, s64 i)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_xchg_acquire(v, i);
> -}
> -
> -static __always_inline s64
> -atomic64_xchg_release(atomic64_t *v, s64 i)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_xchg_release(v, i);
> -}
> -
> -static __always_inline s64
> -atomic64_xchg_relaxed(atomic64_t *v, s64 i)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_xchg_relaxed(v, i);
> -}
> -
> -static __always_inline s64
> -atomic64_cmpxchg(atomic64_t *v, s64 old, s64 new)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline s64
> -atomic64_cmpxchg_acquire(atomic64_t *v, s64 old, s64 new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline s64
> -atomic64_cmpxchg_release(atomic64_t *v, s64 old, s64 new)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline s64
> -atomic64_cmpxchg_relaxed(atomic64_t *v, s64 old, s64 new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic64_try_cmpxchg(atomic64_t *v, s64 *old, s64 new)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic64_try_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic64_try_cmpxchg_acquire(atomic64_t *v, s64 *old, s64 new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic64_try_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic64_try_cmpxchg_release(atomic64_t *v, s64 *old, s64 new)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic64_try_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic64_try_cmpxchg_relaxed(atomic64_t *v, s64 *old, s64 new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic64_try_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic64_sub_and_test(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_sub_and_test(i, v);
> -}
> -
> -static __always_inline bool
> -atomic64_dec_and_test(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_and_test(v);
> -}
> -
> -static __always_inline bool
> -atomic64_inc_and_test(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_and_test(v);
> -}
> -
> -static __always_inline bool
> -atomic64_add_negative(s64 i, atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_add_negative(i, v);
> -}
> -
> -static __always_inline s64
> -atomic64_fetch_add_unless(atomic64_t *v, s64 a, s64 u)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_fetch_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -atomic64_add_unless(atomic64_t *v, s64 a, s64 u)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -atomic64_inc_not_zero(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_not_zero(v);
> -}
> -
> -static __always_inline bool
> -atomic64_inc_unless_negative(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_inc_unless_negative(v);
> -}
> -
> -static __always_inline bool
> -atomic64_dec_unless_positive(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_unless_positive(v);
> -}
> -
> -static __always_inline s64
> -atomic64_dec_if_positive(atomic64_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic64_dec_if_positive(v);
> -}
> -
> -static __always_inline long
> -atomic_long_read(const atomic_long_t *v)
> -{
> -	instrument_atomic_read(v, sizeof(*v));
> -	return arch_atomic_long_read(v);
> -}
> -
> -static __always_inline long
> -atomic_long_read_acquire(const atomic_long_t *v)
> -{
> -	instrument_atomic_read(v, sizeof(*v));
> -	return arch_atomic_long_read_acquire(v);
> -}
> -
> -static __always_inline void
> -atomic_long_set(atomic_long_t *v, long i)
> -{
> -	instrument_atomic_write(v, sizeof(*v));
> -	arch_atomic_long_set(v, i);
> -}
> -
> -static __always_inline void
> -atomic_long_set_release(atomic_long_t *v, long i)
> -{
> -	kcsan_release();
> -	instrument_atomic_write(v, sizeof(*v));
> -	arch_atomic_long_set_release(v, i);
> -}
> -
> -static __always_inline void
> -atomic_long_add(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_add(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_add_return(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_add_return(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_add_return_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_add_return_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_add_return_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_add_return_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_add_return_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_add_return_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_add(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_add(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_add_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_add_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_add_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_add_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_add_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_add_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_long_sub(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_sub(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_sub_return(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_sub_return(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_sub_return_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_sub_return_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_sub_return_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_sub_return_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_sub_return_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_sub_return_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_sub(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_sub(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_sub_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_sub_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_sub_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_sub_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_sub_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_sub_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_long_inc(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_inc(v);
> -}
> -
> -static __always_inline long
> -atomic_long_inc_return(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_return(v);
> -}
> -
> -static __always_inline long
> -atomic_long_inc_return_acquire(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_return_acquire(v);
> -}
> -
> -static __always_inline long
> -atomic_long_inc_return_release(atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_return_release(v);
> -}
> -
> -static __always_inline long
> -atomic_long_inc_return_relaxed(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_return_relaxed(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_inc(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_inc(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_inc_acquire(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_inc_acquire(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_inc_release(atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_inc_release(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_inc_relaxed(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_inc_relaxed(v);
> -}
> -
> -static __always_inline void
> -atomic_long_dec(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_dec(v);
> -}
> -
> -static __always_inline long
> -atomic_long_dec_return(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_return(v);
> -}
> -
> -static __always_inline long
> -atomic_long_dec_return_acquire(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_return_acquire(v);
> -}
> -
> -static __always_inline long
> -atomic_long_dec_return_release(atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_return_release(v);
> -}
> -
> -static __always_inline long
> -atomic_long_dec_return_relaxed(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_return_relaxed(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_dec(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_dec(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_dec_acquire(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_dec_acquire(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_dec_release(atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_dec_release(v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_dec_relaxed(atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_dec_relaxed(v);
> -}
> -
> -static __always_inline void
> -atomic_long_and(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_and(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_and(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_and(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_and_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_and_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_and_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_and_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_and_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_and_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_long_andnot(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_andnot(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_andnot(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_andnot(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_andnot_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_andnot_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_andnot_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_andnot_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_andnot_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_andnot_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_long_or(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_or(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_or(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_or(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_or_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_or_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_or_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_or_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_or_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_or_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -atomic_long_xor(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	arch_atomic_long_xor(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_xor(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_xor(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_xor_acquire(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_xor_acquire(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_xor_release(long i, atomic_long_t *v)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_xor_release(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_xor_relaxed(long i, atomic_long_t *v)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_xor_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_xchg(atomic_long_t *v, long i)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_xchg(v, i);
> -}
> -
> -static __always_inline long
> -atomic_long_xchg_acquire(atomic_long_t *v, long i)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_xchg_acquire(v, i);
> -}
> -
> -static __always_inline long
> -atomic_long_xchg_release(atomic_long_t *v, long i)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_xchg_release(v, i);
> -}
> -
> -static __always_inline long
> -atomic_long_xchg_relaxed(atomic_long_t *v, long i)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_xchg_relaxed(v, i);
> -}
> -
> -static __always_inline long
> -atomic_long_cmpxchg(atomic_long_t *v, long old, long new)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline long
> -atomic_long_cmpxchg_acquire(atomic_long_t *v, long old, long new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline long
> -atomic_long_cmpxchg_release(atomic_long_t *v, long old, long new)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline long
> -atomic_long_cmpxchg_relaxed(atomic_long_t *v, long old, long new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_long_try_cmpxchg(atomic_long_t *v, long *old, long new)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_long_try_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_long_try_cmpxchg_acquire(atomic_long_t *v, long *old, long new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_long_try_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_long_try_cmpxchg_release(atomic_long_t *v, long *old, long new)
> -{
> -	kcsan_release();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_long_try_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_long_try_cmpxchg_relaxed(atomic_long_t *v, long *old, long new)
> -{
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	instrument_atomic_read_write(old, sizeof(*old));
> -	return arch_atomic_long_try_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -atomic_long_sub_and_test(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_sub_and_test(i, v);
> -}
> -
> -static __always_inline bool
> -atomic_long_dec_and_test(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_and_test(v);
> -}
> -
> -static __always_inline bool
> -atomic_long_inc_and_test(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_and_test(v);
> -}
> -
> -static __always_inline bool
> -atomic_long_add_negative(long i, atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_add_negative(i, v);
> -}
> -
> -static __always_inline long
> -atomic_long_fetch_add_unless(atomic_long_t *v, long a, long u)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_fetch_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -atomic_long_add_unless(atomic_long_t *v, long a, long u)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -atomic_long_inc_not_zero(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_not_zero(v);
> -}
> -
> -static __always_inline bool
> -atomic_long_inc_unless_negative(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_inc_unless_negative(v);
> -}
> -
> -static __always_inline bool
> -atomic_long_dec_unless_positive(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_unless_positive(v);
> -}
> -
> -static __always_inline long
> -atomic_long_dec_if_positive(atomic_long_t *v)
> -{
> -	kcsan_mb();
> -	instrument_atomic_read_write(v, sizeof(*v));
> -	return arch_atomic_long_dec_if_positive(v);
> -}
> -
> -#define xchg(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_mb(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_xchg(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define xchg_acquire(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_xchg_acquire(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define xchg_release(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_release(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_xchg_release(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define xchg_relaxed(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_xchg_relaxed(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_mb(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg_acquire(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg_acquire(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg_release(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_release(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg_relaxed(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg_relaxed(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg64(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_mb(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg64(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg64_acquire(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg64_acquire(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg64_release(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_release(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg64_release(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg64_relaxed(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg64_relaxed(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define try_cmpxchg(ptr, oldp, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	typeof(oldp) __ai_oldp = (oldp); \
> -	kcsan_mb(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \
> -	arch_try_cmpxchg(__ai_ptr, __ai_oldp, __VA_ARGS__); \
> -})
> -
> -#define try_cmpxchg_acquire(ptr, oldp, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	typeof(oldp) __ai_oldp = (oldp); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \
> -	arch_try_cmpxchg_acquire(__ai_ptr, __ai_oldp, __VA_ARGS__); \
> -})
> -
> -#define try_cmpxchg_release(ptr, oldp, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	typeof(oldp) __ai_oldp = (oldp); \
> -	kcsan_release(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \
> -	arch_try_cmpxchg_release(__ai_ptr, __ai_oldp, __VA_ARGS__); \
> -})
> -
> -#define try_cmpxchg_relaxed(ptr, oldp, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	typeof(oldp) __ai_oldp = (oldp); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	instrument_atomic_write(__ai_oldp, sizeof(*__ai_oldp)); \
> -	arch_try_cmpxchg_relaxed(__ai_ptr, __ai_oldp, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg_local(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg_local(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg64_local(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_cmpxchg64_local(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define sync_cmpxchg(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_mb(); \
> -	instrument_atomic_write(__ai_ptr, sizeof(*__ai_ptr)); \
> -	arch_sync_cmpxchg(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#define cmpxchg_double(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	kcsan_mb(); \
> -	instrument_atomic_write(__ai_ptr, 2 * sizeof(*__ai_ptr)); \
> -	arch_cmpxchg_double(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -
> -#define cmpxchg_double_local(ptr, ...) \
> -({ \
> -	typeof(ptr) __ai_ptr = (ptr); \
> -	instrument_atomic_write(__ai_ptr, 2 * sizeof(*__ai_ptr)); \
> -	arch_cmpxchg_double_local(__ai_ptr, __VA_ARGS__); \
> -})
> -
> -#endif /* _LINUX_ATOMIC_INSTRUMENTED_H */
> -// 87c974b93032afd42143613434d1a7788fa598f9
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#include <generated/atomic-instrumented.h>
> diff --git a/include/linux/atomic/atomic-long.h b/include/linux/atomic/atomic-long.h
> index 800b8c35992d..2c2986eb66e6 100644
> --- a/include/linux/atomic/atomic-long.h
> +++ b/include/linux/atomic/atomic-long.h
> @@ -1,1014 +1,2 @@
> -// SPDX-License-Identifier: GPL-2.0
> -
> -// Generated by scripts/atomic/gen-atomic-long.sh
> -// DO NOT MODIFY THIS FILE DIRECTLY
> -
> -#ifndef _LINUX_ATOMIC_LONG_H
> -#define _LINUX_ATOMIC_LONG_H
> -
> -#include <linux/compiler.h>
> -#include <asm/types.h>
> -
> -#ifdef CONFIG_64BIT
> -typedef atomic64_t atomic_long_t;
> -#define ATOMIC_LONG_INIT(i)		ATOMIC64_INIT(i)
> -#define atomic_long_cond_read_acquire	atomic64_cond_read_acquire
> -#define atomic_long_cond_read_relaxed	atomic64_cond_read_relaxed
> -#else
> -typedef atomic_t atomic_long_t;
> -#define ATOMIC_LONG_INIT(i)		ATOMIC_INIT(i)
> -#define atomic_long_cond_read_acquire	atomic_cond_read_acquire
> -#define atomic_long_cond_read_relaxed	atomic_cond_read_relaxed
> -#endif
> -
> -#ifdef CONFIG_64BIT
> -
> -static __always_inline long
> -arch_atomic_long_read(const atomic_long_t *v)
> -{
> -	return arch_atomic64_read(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_read_acquire(const atomic_long_t *v)
> -{
> -	return arch_atomic64_read_acquire(v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_set(atomic_long_t *v, long i)
> -{
> -	arch_atomic64_set(v, i);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_set_release(atomic_long_t *v, long i)
> -{
> -	arch_atomic64_set_release(v, i);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_add(long i, atomic_long_t *v)
> -{
> -	arch_atomic64_add(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_add_return(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_add_return_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_add_return_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_add_return_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_add(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_add_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_add_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_add_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_sub(long i, atomic_long_t *v)
> -{
> -	arch_atomic64_sub(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_sub_return(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_sub_return_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_sub_return_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_sub_return_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_sub(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_sub_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_sub_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_sub_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_inc(atomic_long_t *v)
> -{
> -	arch_atomic64_inc(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_return(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_return_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return_release(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_return_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_return_relaxed(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_inc(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_inc_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc_release(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_inc_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_inc_relaxed(v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_dec(atomic_long_t *v)
> -{
> -	arch_atomic64_dec(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_return(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_return_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return_release(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_return_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_return_relaxed(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_dec(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_dec_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec_release(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_dec_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_dec_relaxed(v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_and(long i, atomic_long_t *v)
> -{
> -	arch_atomic64_and(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_and(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_and_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_and_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_and_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_andnot(long i, atomic_long_t *v)
> -{
> -	arch_atomic64_andnot(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_andnot(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_andnot_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_andnot_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_andnot_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_or(long i, atomic_long_t *v)
> -{
> -	arch_atomic64_or(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_or(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_or_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_or_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_or_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_xor(long i, atomic_long_t *v)
> -{
> -	arch_atomic64_xor(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_xor(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_xor_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_xor_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_fetch_xor_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg(atomic_long_t *v, long i)
> -{
> -	return arch_atomic64_xchg(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg_acquire(atomic_long_t *v, long i)
> -{
> -	return arch_atomic64_xchg_acquire(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg_release(atomic_long_t *v, long i)
> -{
> -	return arch_atomic64_xchg_release(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg_relaxed(atomic_long_t *v, long i)
> -{
> -	return arch_atomic64_xchg_relaxed(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic64_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg_acquire(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic64_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg_release(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic64_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg_relaxed(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic64_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic64_try_cmpxchg(v, (s64 *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg_acquire(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic64_try_cmpxchg_acquire(v, (s64 *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg_release(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic64_try_cmpxchg_release(v, (s64 *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg_relaxed(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic64_try_cmpxchg_relaxed(v, (s64 *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_sub_and_test(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_sub_and_test(i, v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_dec_and_test(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_and_test(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_inc_and_test(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_and_test(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_add_negative(long i, atomic_long_t *v)
> -{
> -	return arch_atomic64_add_negative(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_unless(atomic_long_t *v, long a, long u)
> -{
> -	return arch_atomic64_fetch_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_add_unless(atomic_long_t *v, long a, long u)
> -{
> -	return arch_atomic64_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_inc_not_zero(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_not_zero(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_inc_unless_negative(atomic_long_t *v)
> -{
> -	return arch_atomic64_inc_unless_negative(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_dec_unless_positive(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_unless_positive(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_if_positive(atomic_long_t *v)
> -{
> -	return arch_atomic64_dec_if_positive(v);
> -}
> -
> -#else /* CONFIG_64BIT */
> -
> -static __always_inline long
> -arch_atomic_long_read(const atomic_long_t *v)
> -{
> -	return arch_atomic_read(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_read_acquire(const atomic_long_t *v)
> -{
> -	return arch_atomic_read_acquire(v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_set(atomic_long_t *v, long i)
> -{
> -	arch_atomic_set(v, i);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_set_release(atomic_long_t *v, long i)
> -{
> -	arch_atomic_set_release(v, i);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_add(long i, atomic_long_t *v)
> -{
> -	arch_atomic_add(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_add_return(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_add_return_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_add_return_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_add_return_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_add_return_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_add(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_add_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_add_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_add_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_sub(long i, atomic_long_t *v)
> -{
> -	arch_atomic_sub(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_sub_return(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_sub_return_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_sub_return_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_sub_return_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_sub_return_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_sub(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_sub_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_sub_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_sub_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_sub_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_inc(atomic_long_t *v)
> -{
> -	arch_atomic_inc(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_return(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_return_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return_release(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_return_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_inc_return_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_return_relaxed(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_inc(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_inc_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc_release(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_inc_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_inc_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_inc_relaxed(v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_dec(atomic_long_t *v)
> -{
> -	arch_atomic_dec(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_return(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_return_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return_release(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_return_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_return_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_return_relaxed(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_dec(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec_acquire(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_dec_acquire(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec_release(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_dec_release(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_dec_relaxed(atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_dec_relaxed(v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_and(long i, atomic_long_t *v)
> -{
> -	arch_atomic_and(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_and(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_and_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_and_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_and_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_and_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_andnot(long i, atomic_long_t *v)
> -{
> -	arch_atomic_andnot(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_andnot(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_andnot_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_andnot_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_andnot_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_andnot_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_or(long i, atomic_long_t *v)
> -{
> -	arch_atomic_or(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_or(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_or_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_or_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_or_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_or_relaxed(i, v);
> -}
> -
> -static __always_inline void
> -arch_atomic_long_xor(long i, atomic_long_t *v)
> -{
> -	arch_atomic_xor(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_xor(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor_acquire(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_xor_acquire(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor_release(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_xor_release(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_xor_relaxed(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_fetch_xor_relaxed(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg(atomic_long_t *v, long i)
> -{
> -	return arch_atomic_xchg(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg_acquire(atomic_long_t *v, long i)
> -{
> -	return arch_atomic_xchg_acquire(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg_release(atomic_long_t *v, long i)
> -{
> -	return arch_atomic_xchg_release(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_xchg_relaxed(atomic_long_t *v, long i)
> -{
> -	return arch_atomic_xchg_relaxed(v, i);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic_cmpxchg(v, old, new);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg_acquire(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic_cmpxchg_acquire(v, old, new);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg_release(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic_cmpxchg_release(v, old, new);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_cmpxchg_relaxed(atomic_long_t *v, long old, long new)
> -{
> -	return arch_atomic_cmpxchg_relaxed(v, old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic_try_cmpxchg(v, (int *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg_acquire(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic_try_cmpxchg_acquire(v, (int *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg_release(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic_try_cmpxchg_release(v, (int *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_try_cmpxchg_relaxed(atomic_long_t *v, long *old, long new)
> -{
> -	return arch_atomic_try_cmpxchg_relaxed(v, (int *)old, new);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_sub_and_test(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_sub_and_test(i, v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_dec_and_test(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_and_test(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_inc_and_test(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_and_test(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_add_negative(long i, atomic_long_t *v)
> -{
> -	return arch_atomic_add_negative(i, v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_fetch_add_unless(atomic_long_t *v, long a, long u)
> -{
> -	return arch_atomic_fetch_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_add_unless(atomic_long_t *v, long a, long u)
> -{
> -	return arch_atomic_add_unless(v, a, u);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_inc_not_zero(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_not_zero(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_inc_unless_negative(atomic_long_t *v)
> -{
> -	return arch_atomic_inc_unless_negative(v);
> -}
> -
> -static __always_inline bool
> -arch_atomic_long_dec_unless_positive(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_unless_positive(v);
> -}
> -
> -static __always_inline long
> -arch_atomic_long_dec_if_positive(atomic_long_t *v)
> -{
> -	return arch_atomic_dec_if_positive(v);
> -}
> -
> -#endif /* CONFIG_64BIT */
> -#endif /* _LINUX_ATOMIC_LONG_H */
> -// e8f0e08ff072b74d180eabe2ad001282b38c2c88
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#include <generated/atomic-long.h>
> diff --git a/include/linux/atomic/atomic-arch-fallback.h b/scripts/atomic/atomic-arch-fallback.h_shipped
> similarity index 99%
> copy from include/linux/atomic/atomic-arch-fallback.h
> copy to scripts/atomic/atomic-arch-fallback.h_shipped
> index 6db58d180866..357627e3e304 100644
> --- a/include/linux/atomic/atomic-arch-fallback.h
> +++ b/scripts/atomic/atomic-arch-fallback.h_shipped
> @@ -2386,4 +2386,3 @@ arch_atomic64_dec_if_positive(atomic64_t *v)
>  #endif
>  
>  #endif /* _LINUX_ATOMIC_FALLBACK_H */
> -// 8e2cc06bc0d2c0967d2f8424762bd48555ee40ae
> diff --git a/include/linux/atomic/atomic-instrumented.h b/scripts/atomic/atomic-instrumented.h_shipped
> similarity index 99%
> copy from include/linux/atomic/atomic-instrumented.h
> copy to scripts/atomic/atomic-instrumented.h_shipped
> index 5d69b143c28e..89cde1f1eb67 100644
> --- a/include/linux/atomic/atomic-instrumented.h
> +++ b/scripts/atomic/atomic-instrumented.h_shipped
> @@ -2045,4 +2045,3 @@ atomic_long_dec_if_positive(atomic_long_t *v)
>  })
>  
>  #endif /* _LINUX_ATOMIC_INSTRUMENTED_H */
> -// 87c974b93032afd42143613434d1a7788fa598f9
> diff --git a/include/linux/atomic/atomic-long.h b/scripts/atomic/atomic-long.h_shipped
> similarity index 99%
> copy from include/linux/atomic/atomic-long.h
> copy to scripts/atomic/atomic-long.h_shipped
> index 800b8c35992d..411d16a7f4eb 100644
> --- a/include/linux/atomic/atomic-long.h
> +++ b/scripts/atomic/atomic-long.h_shipped
> @@ -1011,4 +1011,3 @@ arch_atomic_long_dec_if_positive(atomic_long_t *v)
>  
>  #endif /* CONFIG_64BIT */
>  #endif /* _LINUX_ATOMIC_LONG_H */
> -// e8f0e08ff072b74d180eabe2ad001282b38c2c88
> diff --git a/scripts/atomic/check-atomics.sh b/scripts/atomic/check-atomics.sh
> deleted file mode 100755
> index 0e7bab3eb0d1..000000000000
> --- a/scripts/atomic/check-atomics.sh
> +++ /dev/null
> @@ -1,33 +0,0 @@
> -#!/bin/sh
> -# SPDX-License-Identifier: GPL-2.0
> -#
> -# Check if atomic headers are up-to-date
> -
> -ATOMICDIR=$(dirname $0)
> -ATOMICTBL=${ATOMICDIR}/atomics.tbl
> -LINUXDIR=${ATOMICDIR}/../..
> -
> -echo '' | sha1sum - > /dev/null 2>&1
> -if [ $? -ne 0 ]; then
> -	printf "sha1sum not available, skipping atomic header checks.\n"
> -	exit 0
> -fi
> -
> -cat <<EOF |
> -linux/atomic/atomic-instrumented.h
> -linux/atomic/atomic-long.h
> -linux/atomic/atomic-arch-fallback.h
> -EOF
> -while read header; do
> -	OLDSUM="$(tail -n 1 ${LINUXDIR}/include/${header})"
> -	OLDSUM="${OLDSUM#// }"
> -
> -	NEWSUM="$(sed '$d' ${LINUXDIR}/include/${header} | sha1sum)"
> -	NEWSUM="${NEWSUM%% *}"
> -
> -	if [ "${OLDSUM}" != "${NEWSUM}" ]; then
> -		printf "warning: generated include/${header} has been modified.\n"
> -	fi
> -done
> -
> -exit 0
> diff --git a/scripts/atomic/gen-atomics.sh b/scripts/atomic/gen-atomics.sh
> index 5b98a8307693..ea90b7ac8f2c 100755
> --- a/scripts/atomic/gen-atomics.sh
> +++ b/scripts/atomic/gen-atomics.sh
> @@ -5,16 +5,12 @@
>  
>  ATOMICDIR=$(dirname $0)
>  ATOMICTBL=${ATOMICDIR}/atomics.tbl
> -LINUXDIR=${ATOMICDIR}/../..
>  
>  cat <<EOF |
> -gen-atomic-instrumented.sh      linux/atomic/atomic-instrumented.h
> -gen-atomic-long.sh              linux/atomic/atomic-long.h
> -gen-atomic-fallback.sh          linux/atomic/atomic-arch-fallback.h
> +gen-atomic-instrumented.sh      atomic-instrumented.h
> +gen-atomic-long.sh              atomic-long.h
> +gen-atomic-fallback.sh          atomic-arch-fallback.h
>  EOF
> -while read script header args; do
> -	/bin/sh ${ATOMICDIR}/${script} ${ATOMICTBL} ${args} > ${LINUXDIR}/include/${header}
> -	HASH="$(sha1sum ${LINUXDIR}/include/${header})"
> -	HASH="${HASH%% *}"
> -	printf "// %s\n" "${HASH}" >> ${LINUXDIR}/include/${header}
> +while read script header; do
> +	/bin/sh ${ATOMICDIR}/${script} ${ATOMICTBL} > ${ATOMICDIR}/${header}_shipped
>  done
> -- 
> 2.32.0
> 

Powered by blists - more mailing lists