Message-Id: <7DD2C38C-CBB3-41EE-A195-50E4C2451B7E@jrtc27.com>
Date:   Mon, 1 May 2023 21:08:48 +0100
From:   Jessica Clarke <jrtc27@...c27.com>
To:     Björn Töpel <bjorn@...nel.org>
Cc:     Heiko Stuebner <heiko@...ech.de>,
        Palmer Dabbelt <palmer@...belt.com>,
        linux-riscv <linux-riscv@...ts.infradead.org>,
        Paul Walmsley <paul.walmsley@...ive.com>,
        Kito Cheng <kito.cheng@...ive.com>,
        Conor Dooley <conor.dooley@...rochip.com>,
        matthias.bgg@...il.com,
        Heinrich Schuchardt <heinrich.schuchardt@...onical.com>,
        Greentime Hu <greentime.hu@...ive.com>, nick.knight@...ive.com,
        christoph.muellner@...ll.eu, philipp.tomsich@...ll.eu,
        richard.henderson@...aro.org, arnd@...db.de,
        linux-kernel@...r.kernel.org,
        Heiko Stuebner <heiko.stuebner@...ll.eu>
Subject: Re: [PATCH 0/4] Expose the isa-string via the AT_BASE_PLATFORM aux
 vector

On 1 May 2023, at 20:55, Björn Töpel <bjorn@...nel.org> wrote:
> 
> Heiko Stuebner <heiko@...ech.de> writes:
> 
>> From: Heiko Stuebner <heiko.stuebner@...ll.eu>
>> 
>> The hwprobing infrastructure was merged recently [0] and contains a
>> mechanism to probe both extensions but also microarchitecural features
>> on a per-core level of detail.
>> 
>> While discussing the solution internally we identified some possible issues,
>> tried to understand the underlying issue and come up with a solution for it.
>> All these deliberations overlapped with hwprobing being merged, but that
>> shouldn't really be an issue, as both have their usability - see below.
>> Also please see the "Things to consider" at the bottom!
>> 
>> 
>> Possible issues:
>> - very much limited to Linux
>> - schedulers run processes on all cores by default, so will need
>>  the common set of extensions in most cases
> 
> ...which hwprobe has support for via the CPU mask. no?
> 
>> - each new extensions requires an uapi change, requiring at least
>>  two pieces of software to be changed
>> - adding another extension requires a review process (only known
>>  extensions can be exposed to user-space)
>> - vendor extensions have special needs and therefore possibly
>>  don’t fit well
>> 
>> 
>> Limited to Linux:
>> -----------------
>> 
>> The syscall and its uapi is Linux-specific and other OSes probably
>> will not defer to our review process and requirements just to get
>> new bits in. Instead most likely they'll build their own systems,
>> leading to fragmentation.
> 
> There are a number of examples where multiple OSs have followed what
> Linux does, and vice versa. I'd say the opposite -- today system
> builders do not do their own solution, but review what's out there and
> mimics existing ones.

Where they can. But if the interface is “make architecture-dependent
syscall X” that’s not going to fly on other OSes where syscalls are not
architecture-dependent. Similarly if it’s “go read auxarg Y” where Y is
architecture-dependent and the OS in question has
architecture-independent auxargs. Or the system doesn’t even have
auxargs. Now, at the end of the day, I couldn’t care less what Linux
does to communicate the information to userspace, what matters is what
the userspace interface is that random IFUNCs are going to make use of.
Something which seems to be woefully lacking from this discussion. Is
the interface going to just be syscall(2)? Or is there going to be some
higher-level interface that other OSes *do* have a hope of being able
to implement?

> Personally I think this argument is moot, and will not matter much for
> fragmentation.
> 
>> Feature on all cores:
>> ---------------------
>> 
>> Arnd previously ([1]) commented in the discussion, that there
>> should not be a need for optimization towards hardware with an
>> asymmetric set of features. We believe so as well, especially
>> when talking about an interface that helps processes to identify
>> the optimized routines they can execute.
>> 
>> Of course seeing it with this finality might not take into account
>> the somewhat special nature of RISC-V, but nevertheless it describes
>> the common case for programs very well.
>> 
>> For starters the scheduler in its default behaviour, will try to use any
>> available core, so the standard program behaviour will always need the
>> intersection of available extensions over all cores.
>> 
>> 
>> Limiting program execution to specific cores will likely always be a
>> special use case and already requires Linux-specific syscalls to
>> select the set of cores.
>> 
>> So while it can come in handy to get per-core information down the road
>> via the hwprobing interface, most programs will just want to know if
>> they can use a extension on just any core.
>> 
>> 
>> Review process:
>> ---------------
>> 
>> There are so many (multi-letter-)extensions already with even more in
>> the pipeline. To expose all of them, each will require a review process
>> and uapi change that will make defining all of them slow as the
>> kernel needs patching after which userspace needs to sync in the new
>> api header.
>> 
>> 
>> Vendor-extensions:
>> ------------------
>> 
>> Vendor extensions are special in their own right.
>> Userspace probably will want to know about them, but we as the kernel
>> don't want to care about them too much (except as errata), as they're
>> not part of the official RISC-V ISA spec.
>> 
>> Getting vendor extensions from the dt to userspace via hwprobe would
>> require coordination efforts and as vendors have the tendency to invent
>> things during their development process before trying to submit changes
>> upstream this likely would result in conflicts with assigned ids down
>> the road. Which in turn then may create compatibility-issues with
>> userspace builds built on top of the mainline kernel or a pre-
>> existing vendor kernel.
>> 
>> The special case also is that vendor A could in theory implement an
>> extension from vendor B. So this would require to actually assign
>> separate hwprobe keys to vendors (key for xthead extensions, key for
>> xventana extensions, etc). This in turn would require vendors to
>> come to the mainline kernel to get assigned a key (which in reality
>> probably won't happen), which would then make the kernel community
>> sort of an id authority.
>> 
>> 
>> 
>> 
>> To address these, the attached patch series adds a second interface
>> for the common case and "just" exposes the isa-string via the
>> AT_BASE_PLATFORM aux vector.
> 
> *A second interface* introduced the second hwprobe landed. Really?
> Start a discussion on how to extend hwprobe instead.

I’ve been trying to push for something other than this for months, but
RVI took no interest in dealing with it until it got closer to these
landing, at which point finally some action was taken. But even then,
heels were dragged, and it took hwprobe being landed to force them to
finally publish things. But of course too late, so now the ecosystem is
forever screwed thanks to their inaction.

All I wanted was some simple extension string -> version number function
as a standardised userspace interface... because at the end of the day
people just want to know “can I use extension Y?”, possibly with a
minimum version. But maybe there’s some hope that Linux libcs will
translate such queries into poking at this hwprobe thing. Though god
knows what they’re going to do about vendor extensions, especially if
those vendor extensions only get supported in vendor forks of the
kernel (who’s allocating their encodings? Xvendorfoo exists to
namespace them and give vendors control...).

Jess

Powered by blists - more mailing lists