Message-ID: <7d8133c7e0186bdaeb3893c1c808148dc0d11945.camel@intel.com>
Date:   Mon, 14 Nov 2022 23:15:44 +0000
From:   "Edgecombe, Rick P" <rick.p.edgecombe@...el.com>
To:     "Torvalds, Linus" <torvalds@...ux-foundation.org>
CC:     "keescook@...omium.org" <keescook@...omium.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "fweimer@...hat.com" <fweimer@...hat.com>,
        "hjl.tools@...il.com" <hjl.tools@...il.com>,
        "x86@...nel.org" <x86@...nel.org>
Subject: CET shadow stack app compatibility

Hi Linus,

Could you weigh in on some brewing ecosystem compatibility issues
around x86 CET[0] shadow stacks? This is the CPU security feature that
keeps a separate protected stack to record return addresses, and
verifies them on return. Support for this feature is not upstream in
the kernel and so the issues discussed here are future problems that
have not happened yet.

The issues all have a root cause of support for CET in tools spreading
widely while kernel support was still in development. This has lead to:
1. Some existing binaries (node.js, PyPy, CRIU) that will break when
glibc updates to use the kernel CET APIs.
2. GCC C++ exception stack unwinding code expecting old development
versions of the kernel ABI.

On the first issue, once there is kernel support, glibc plans to
immediately update in such a way that some existing distro binaries
will break against it. So the scenario is existing distro binaries
being used with future versions of glibc. The known extent of breakage
is limited to some packages of node.js and PyPy, and any version of
CRIU, but it’s reasonable to assume that there are undetected breakages
based on how it came about.

The breakage derives from how the decision is made on whether to enable
shadow stack enforcement. Glibc will do this by checking a bit in the
elf header of the binary. It then tells the kernel to turn CET on via a
separate kernel API. But instead of this elf bit being selected by
application developers, it was mostly applied in various automated ways
(mostly default on) by distro builds for years. This huge amount of
untested enablement has not generated any visible issues for users yet,
because without kernel support the presence of this bit has not
generated any actual CET enforcement.

In some ways it is a variation of past compatibility problems around
distros overriding package defaults for compiler hardening. But the
difference is that the kernel support is involved in doing the
enforcement in this case, leading to the issues going undetected.

For the second issue, there are also problems lurking in gcc. The gcc
CET support has preceded the kernel changes and the unwinding code
assumes things about the kernel shadow stack signal frame ABI that have
changed over the course of CET kernel development. It is compatible by
luck for now, but old GGC’s that apply the existing elf bit (going back
to gcc-8) can generate future binaries that would constrain the shadow
stack signal frame from expanding, which there are already plans to do.

I would like to make this go smoother all around by having the kernel
detect the existing elf bit and refuse to enable CET for these
applications, like this[1]. Then the binaries derived from the pre-
kernel support era would just continue to run normally without CET
enforcement. The intention would be to force tools to pick a new elf
bit to denote compatibility for this feature. With a tools reset, this
time the upstream kernel would have shadow stack support ahead of tools
and so any issues would likely show up earlier.

The best place to exclude the old binaries from shadow stack support
would be in the glibc loader, but developers of that (on CC) are
against creating new CET elf bits. So the kernel would be taking a
stand here and would essentially burn this bit from the kernel side.

Are you generally ok with the kernel reaching out and getting involved
in this shadow stack enablement decision like this?

Thanks,

Rick

[0] https://lwn.net/Articles/885220/
[1] 
https://lore.kernel.org/lkml/20221104223604.29615-38-rick.p.edgecombe@intel.com/

Powered by blists - more mailing lists