Message-ID: <aNcAela5tln5KTUI@debug.ba.rivosinc.com>
Date: Fri, 26 Sep 2025 14:07:06 -0700
From: Deepak Gupta <debug@...osinc.com>
To: Charles Mirabile <cmirabil@...hat.com>
Cc: pjw@...nel.org, Liam.Howlett@...cle.com, a.hindborg@...nel.org,
	akpm@...ux-foundation.org, alex.gaynor@...il.com,
	alexghiti@...osinc.com, aliceryhl@...gle.com,
	alistair.francis@....com, andybnac@...il.com, aou@...s.berkeley.edu,
	arnd@...db.de, atishp@...osinc.com, bjorn3_gh@...tonmail.com,
	boqun.feng@...il.com, bp@...en8.de, brauner@...nel.org,
	broonie@...nel.org, charlie@...osinc.com, cleger@...osinc.com,
	conor+dt@...nel.org, conor@...nel.org, corbet@....net,
	dave.hansen@...ux.intel.com, david@...hat.com,
	devicetree@...r.kernel.org, ebiederm@...ssion.com,
	evan@...osinc.com, gary@...yguo.net, hpa@...or.com,
	jannh@...gle.com, jim.shu@...ive.com, kees@...nel.org,
	kito.cheng@...ive.com, krzk+dt@...nel.org,
	linux-arch@...r.kernel.org, linux-doc@...r.kernel.org,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-kselftest@...r.kernel.org, linux-mm@...ck.org,
	linux-riscv@...ts.infradead.org, lorenzo.stoakes@...cle.com,
	lossin@...nel.org, mingo@...hat.com, ojeda@...nel.org,
	oleg@...hat.com, palmer@...belt.com, paul.walmsley@...ive.com,
	peterz@...radead.org, richard.henderson@...aro.org,
	rick.p.edgecombe@...el.com, robh@...nel.org,
	rust-for-linux@...r.kernel.org, samitolvanen@...gle.com,
	shuah@...nel.org, tglx@...utronix.de, tmgross@...ch.edu,
	vbabka@...e.cz, x86@...nel.org, zong.li@...ive.com
Subject: Re: [PATCH v19 00/27] riscv control-flow integrity for usermode

On Fri, Sep 26, 2025 at 04:28:58PM -0400, Charles Mirabile wrote:
>Hi Deepak -
>
>On Fri, Sep 26, 2025 at 3:57 PM Deepak Gupta <debug@...osinc.com> wrote:
>>
>> Hi Charles,
>>
>> Thanks for response. Rest inline
>>
>> On Fri, Sep 26, 2025 at 03:29:19PM -0400, Charles Mirabile wrote:
>> >Hi -
>> >
>> >Hoping that I got everything right with git-send-email so that this is
>> >delivered alright...
>> >
>> >Wanted to jump in to head off a potential talking past one another /
>> >miscommunication situation I see here.
>> >
>> >On Wed, Sep 24, 2025 at 08:36:11AM -0600, Paul Walmsley wrote:
>> >> Hi,
>> >>
>> >> On Thu, 31 Jul 2025, Deepak Gupta wrote:
>> >>
>> >> [ ... ]
>> >>
>> >> > vDSO related Opens (in the flux)
>> >> > =================================
>> >> >
>> >> > I am listing these opens for laying out plan and what to expect in future
>> >> > patch sets. And of course for the sake of discussion.
>> >> >
>> >>
>> >> [ ... ]
>> >>
>> >> > How many vDSOs
>> >> > ---------------
>> >> > Shadow stack instructions are carved out of zimop (may be operations) and if CPU
>> >> > doesn't implement zimop, they're illegal instructions. Kernel could be running on
>> >> > a CPU which may or may not implement zimop. And thus kernel will have to carry 2
>> >> > different vDSOs and expose the appropriate one depending on whether CPU implements
>> >> > zimop or not.
>> >>
>> >> If we merge this series without this, then when CFI is enabled in the
>> >> Kconfig, we'll wind up with a non-portable kernel that won't run on older
>> >> hardware.  We go to great lengths to enable kernel binary portability
>> >> across the presence or absence of other RISC-V extensions, and I think
>> >> these CFI extensions should be no different.
>> >
>> >That is not true, this series does not contain the VDSO changes so it can
>> >be merged as is.
>>
>> Look at patch 23/27. It does have vDSO change. Although shadow stack
>> instruction are inserted as compiled flag for vDSO only when cfi config is
>> selected by user. Right now default is "No". So it won't impact anyone unles
>> user explicitly says "Yes".
>
>Yes sorry I caught that after hitting send and replied to my own email
>(but then I said 19/27 instead of 23/27 *facepalm*)
>
>>
>> >
>> >>
>> >> So before considering this for merging, I'd like to see at least an
>> >> attempt to implement the dual-vDSO approach (or something equivalent)
>> >> where the same kernel binary with CFI enabled can run on both pre-Zimop
>> >> and post-Zimop hardware, with the existing userspaces that are common
>> >> today.
>> >
>> >I agree that when the VDSO patches are submitted for inclusion they should
>> >be written in a way that avoids limiting the entire kernel to either
>> >pre-Zimop or post-Zimop hardware based on the config, but I think it
>> >should be quite possible to perform e.g. runtime patching of the VDSO
>> >to replace the Zimop instructions with nops if the config is enabled but
>> >the hardware does not support Zimop.
>>
>> Why kernel need to do this extra work of carry two binaries and patching it
>> runtime?
>>
>> If for instance we do this, and then this allow this kernel to be taken to
>> pre-Zimop hardware, it is assumed that entire userspace for such hardware
>> was compiled without shadow stack (thus no zimop). In that case, kernel
>> should have been compiled without CFI option.
>
>You raise a good point, it just breaks the tradition of runtime
>detection and backwards compat that has been the standard for riscv
>extensions in the kernel so far.

riscv (and others arches) have been able to do that because of "alternatives".
It's just that due to composable nature of riscv, alternatives are just spread
everywhere in the code and feels like riscv is doing something unique here.
Whenever there is a surgical placement of certain instructions in kernel, it
could be hidden behind alternatives and be patched in runtime.

However situations where instructions are emitted as part of codegen, there is
no hiding. Either it works or it doesn't. If we have auto vectorization enabled
in usermode, such a binary won't run on hardware which doesn't implement vector.

In case of shadow stack, it similar situation. If enabled compiler decides to
insert sspush and sspopchk. They necessarily won't be prologue or epilogue but
somewhere in function body as deemed fit by compiler, thus increasing the
complexity of runtime patching.

More so, here are wishing for kernel to do this patching for usermode vDSO when
there is no guarantee of such of rest of usermode (which if was compiled with
shadow stack would have faulted before vDSO's sspush/sspopchk if ran on
pre-zimop hardware)

>
>It would be nice if a kernel could be built that would run on both
>pre-Zimop and post-Zimop hardware and be able to offer CFI to
>userspace when running on hardware with Zimop (and Zicfiss / Zicfilp)
>but agree that it is a burden.
>
>>
>> Just for sake of thought exercise, let's say Fedora 43 is first release with
>> RVA23 compatiblity (zimop and shadow stack), there is no way this and future
>> release will be able to run on pre-zimop hardware. Unless redhat is going to
>> start two different binary distribution. One for pre-zimop and one for
>> post-zimop. If that would be the case, then compiling two different kernel for
>> such two different hardware would be least of the worry.
>
>It would be one thing if there were hardware supporting
>Zimop/Zicfiss/Zicfilp readily available, but I am not aware of any

And that's the reason currently default is "No" for cfi config in kernel.
Hope is whenever we have hardware, we can start lighting up and take decision
how to proceed. I keep reiterting, door isn't closed yet but we gotta approach
the door.

>platform other than qemu to test this code. 

I have tested this with qemu in following configurations

qemu implements cfi extensions:
Kernel with cfi enable is able to host userspace with and without cfi compiled.
Kernel with cfi enable is able to host userspace with cfi but disabled (default
to zimop)
Kernel without cfi enable is able to host userspace with cfi (default to zimop)
and without cfi.

qemu doesn't implement cfi extension:
- Kernel without cfi enable is able to host userspace without cfi compiled in.
- Kernel without cfi enable hosting userspace with cfi compiled in faults in
   libc/ld on first sspush.
- Kernel with cfi enable trying to host userspace with cfi faults in libc/ld on
   first sspush.
- Kernel with cfi enable trying to host userspace without cfi faults in vDSO on
   first sspush.

Last case is the only breaking case, rest all compatibilities stories work.
In order to solve this last compatiblity story, I am fearful that we might be
adding un-necessary complexity in kernel which isn't desired because rest of
the userspace won't be signing up for that complexity of patching and making
it work with single binary.

> Since it breaks
>compatibility with hardware I am not sure anyone will be able to do
>anything with this config option and it moves the burden on to each
>distro to go in and specifically enabling it vs just making things
>work to get important security improvements if the hardware has
>support and not if it doesn't in a backwards compatible way.

I wished that shadow stack instructions came out of HINT space. But it is
what it is. Perhaps distro should give this feedback to RVI. But here we
are.

zimop is backward compatible only RVA23 onwards. That's why it's important
for distro to make a decision on this. Once they compile for RVA23 profile,
it assumed a clean break from previous hardware. Future extensions will also
take encodings out of zimop and thus I believe its better to take that decision
now with first user of zimop.

>
>>
>> Only other usecase is of a seasoned kernel developer or build your own stuff
>> in embedded environment, those users can anyways are advanced users. But it
>> forces complexity on rest of kernel. There will be more extensions taking zimop
>> encodings in future, we will end up patching vDSO and keep this complexity
>> while rest of the userspace will not be patched and will be separate binary
>> distribution (if OS distros endup distributing multiple binaries per release)
>>
>> >
>> >However, that concern should not hold up this patch series. Raise it again
>> >when the VDSO patches are posted.
>>
>> As I said earlier, these changes default cfi config to No. So whenever this
>> is selected "Yes" by a distro, they can drive such patches (if there is a real
>> need)
>
>If we did the patching we could make this config default to yes to
>that you are building a kernel that is set up to be able to offer CFI
>when running on hardware which supports it as long as you have a
>toolchain that recognizes the extensions which I think would be good
>for moving this important security feature forward.

Again, Why kernel should be doing this when rest of the userspace isn't
patchable on pre-zimop hardware (thats the only scenario patching is needed)?

Are distro's distributing different binary for with autovec and without autovec
for different kind of riscv hardware?

Giving example of Fedora 43, once it is compiled in with cfi enabling, kernel is
also compiled in with the feature. Its not like there is going to "Fedora
43_rv64gc" release. If there is going to be a "Fedora 43_rv64gc" release, it'll
be much easier to no select CFI for that release's kernel compile rather than
kernel doing patching in runtime (rest of userspace is not doing any patching)

>
>>
>> >
>> >>
>> >> thanks Deepak,
>> >>
>> >> - Paul
>> >
>> >Best - Charlie
>> >
>>
>
>Sorry for stirring the pot on this. I really appreciate your work on
>this patch series.
>
>I agree that this is a difficult call, and I could see it going either
>way but I lean towards trying to maintain the backwards compatibility
>because the hardware doesn't exist yet.
>
>Best - Charlie
>

Powered by blists - more mailing lists