Message-ID: <20230828-deserve-paging-25666290e312@spud>
Date:   Mon, 28 Aug 2023 18:08:40 +0100
From:   Conor Dooley <conor@...nel.org>
To:     Charlie Jenkins <charlie@...osinc.com>
Cc:     Palmer Dabbelt <palmer@...belt.com>,
        linux-riscv@...ts.infradead.org, linux-kernel@...r.kernel.org,
        Paul Walmsley <paul.walmsley@...ive.com>, aou@...s.berkeley.edu
Subject: Re: [PATCH 1/5] riscv: Checksum header

On Mon, Aug 28, 2023 at 09:55:49AM -0700, Charlie Jenkins wrote:
> On Sun, Aug 27, 2023 at 01:25:27PM +0100, Conor Dooley wrote:
> > On Sun, Aug 27, 2023 at 11:28:33AM +0100, Conor Dooley wrote:
> > > On Sat, Aug 26, 2023 at 07:00:47PM -0700, Palmer Dabbelt wrote:
> > > > On Sat, 26 Aug 2023 18:42:41 PDT (-0700), Conor Dooley wrote:
> > > > > On Sat, Aug 26, 2023 at 06:26:06PM -0700, Charlie Jenkins wrote:
> > > > > > Provide checksum algorithms that have been designed to leverage riscv
> > > > > > instructions such as rotate. In 64-bit, can take advantage of the larger
> > > > > > register to avoid some overflow checking.
> > > > > > 
> > > > > > Add configuration for Zba extension and add march for Zba and Zbb.
> > > > > > 
> > > > > > Signed-off-by: Charlie Jenkins <charlie@...osinc.com>
> > > > > > ---
> > > > > >  arch/riscv/Kconfig                | 23 +++++++++++
> > > > > >  arch/riscv/Makefile               |  2 +
> > > > > >  arch/riscv/include/asm/checksum.h | 86 +++++++++++++++++++++++++++++++++++++++
> > > > > >  3 files changed, 111 insertions(+)
> > > > > > 
> > > > > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > > > > > index 4c07b9189c86..8d7e475ca28d 100644
> > > > > > --- a/arch/riscv/Kconfig
> > > > > > +++ b/arch/riscv/Kconfig
> > > > > > @@ -507,6 +507,29 @@ config RISCV_ISA_V_DEFAULT_ENABLE
> > > > > >  	  If you don't know what to do here, say Y.
> > > > > > +config TOOLCHAIN_HAS_ZBA
> > > > > > +	bool
> > > > > > +	default y
> > > > > > +	depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zba)
> > > > > > +	depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zba)
> > > > > > +	depends on LLD_VERSION >= 150000 || LD_VERSION >= 23900
> > > > > > +	depends on AS_HAS_OPTION_ARCH
> > > > > > +
> > > > > > +config RISCV_ISA_ZBA
> > > > > > +	bool "Zba extension support for bit manipulation instructions"
> > > > > > +	depends on TOOLCHAIN_HAS_ZBA
> > > > > > +	depends on MMU
> > > > > > +	depends on RISCV_ALTERNATIVE
> > > > > > +	default y
> > > > > > +	help
> > > > > > +	   Adds support to dynamically detect the presence of the ZBA
> > > > > > +	   extension (basic bit manipulation) and enable its usage.
> > > > > > +
> > > > > > +	   The Zba extension provides instructions to accelerate a number
> > > > > > +	   of bit-specific address creation operations.
> > > > > > +
> > > > > > +	   If you don't know what to do here, say Y.
> > > > > > +
> > > > > >  config TOOLCHAIN_HAS_ZBB
> > > > > >  	bool
> > > > > >  	default y
> > > > > > diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> > > > > > index 6ec6d52a4180..51fa3f67fc9a 100644
> > > > > > --- a/arch/riscv/Makefile
> > > > > > +++ b/arch/riscv/Makefile
> > > > > > @@ -61,6 +61,8 @@ riscv-march-$(CONFIG_ARCH_RV64I)	:= rv64ima
> > > > > >  riscv-march-$(CONFIG_FPU)		:= $(riscv-march-y)fd
> > > > > >  riscv-march-$(CONFIG_RISCV_ISA_C)	:= $(riscv-march-y)c
> > > > > >  riscv-march-$(CONFIG_RISCV_ISA_V)	:= $(riscv-march-y)v
> > > > > > +riscv-march-$(CONFIG_RISCV_ISA_ZBA)	:= $(riscv-march-y)_zba
> > > > > > +riscv-march-$(CONFIG_RISCV_ISA_ZBB)	:= $(riscv-march-y)_zbb
> > > > > 
> > > > > AFAICT, this is going to break immediately on any system that enables
> > > > > RISCV_ISA_ZBA (which will happen by default) but does not support the
> > > > > extension. You made the option depend on RISCV_ALTERNATIVE, but I do
> > > > > not see any use of alternatives in the code to actually perform the
> > > > > dynamic detection of Zba.
> > > > 
> > > > I guess we kind of have an ambiguity here: for stuff like C we just
> > > > unconditionally use the instructions, but for the rest we probe first.  We
> > > > should probably have three states for each extension: disabled, dynamically
> > > > detected, and assumed.
> > > 
> > > You mean, just add some comments to the makefile surrounding each
> > > section or to some rst documentation?
> > 
> > Also, the code here doesn't build w/
> > 	warning: invalid argument to '-march': '_zba_zbb_zicsr_zifencei_zihintpause'
> > so there's something else wrong with TOOLCHAIN_HAS_ZBA :)
> It is odd that this is missing 'rv64ima' or 'rv32ima' at the beginning of
> this string. What configuration are you using that could cause that to
> be left off?

I don't know, but that configuration is pretty pervasive. The patchwork
CI blew up too & that is using kernel.org toolchains built by Arnd:
https://mirrors.edge.kernel.org/pub/tools/crosstool/

> Compiling with defconfig automatically enables Zba and appears to not
> cause this issue. I realized that I put the header definitions for
> do_csum and csum_ipv6_magic in this patch instead of the next one so the
> code will fail to compile from this but not due to march settings.
> > 
> > > 
> > > > > Note that for fd & v, we add it to riscv-march-y, but then immediately
> > > > > remove it again before passing to the compiler, only allow them in
> > > > > AFLAGS:
> > > > > 	# Remove F,D,V from isa string for all. Keep extensions between "fd" and "v" by
> > > > > 	# matching non-v and non-multi-letter extensions out with the filter ([^v_]*)
> > > > > 	KBUILD_CFLAGS += -march=$(shell echo $(riscv-march-y) | sed -E 's/(rv32ima|rv64ima)fd([^v_]*)v?/\1\2/')
> > > > > 
> > > > > What am I missing?
> > > > 
> > > > FD and V both have state that can be saved lazily, so we can't let arbitrary
> > > > code use them.  The extensions formally known as B don't add state, so they
> > > > are safe to flip on in arbitrary places (aside from the issues you pointed
> > > > out above).
> > > 
> > > I probably went about this badly since you missed the point. I was
> > > trying to point out that for anything other than the compressed
> > > extensions in the block above that we only pass them in march to the
> > > assembler, and not to the compiler, in contrast to this patch which just
> > > always passes them. I should have pointed to how we handled the
> > > in-kernel Zbb stuff & asked how this was any different, would probably
> > > have been clearer.
> > > 
> I supposed it might be better if I submit these changes in a different
> patch so we can have more discussion there. Zbb was previously only used
> by assembly files (arch/riscv/lib/strcmp.S, arch/riscv/lib/strlen.S,
> arch/riscv/lib/strncmp.S). I wanted to add them to the compiler so that
> that C programs could leverage these extensions. However, I neglected to
> consider machines that compile the kernel with these extensions but have
> cores without these extensions.

Less so cores, since we don't support heterogeneous stuff, and moreso
platforms that do not support the extensions. It's expected that the
same kernel could in theory be used across a wide variety of systems.

> The purpose of using these extensions is
> to save a couple of clock cycles, so if it is necessary to first
> check if the extension is enabled it may not be worth it for these
> functions.

That's still possible, it's what the alternatives mechanism exists for.
During boot the codepaths are patched to use what works for a given
machine, check out the code that makes use of Zbb or
riscv_has_extension_[un]likely(). You'd need to do something like the
existing users of Zbb instructions does, with an alternative used to
avoid the custom asm implementations when the hardware does not support
them. (That's what the CONFIG_ALTERNATIVE & CONFIG_AS_HAS_OPTION_ARCH
options you made the TOOLCHAIN_HAS_ZBA depend on are for).

Download attachment "signature.asc" of type "application/pgp-signature" (229 bytes)

Powered by blists - more mailing lists