Date:   Tue, 15 Nov 2022 16:57:16 +0000
From:   "Edgecombe, Rick P" <rick.p.edgecombe@...el.com>
To:     "Torvalds, Linus" <torvalds@...ux-foundation.org>,
        "fweimer@...hat.com" <fweimer@...hat.com>
CC:     "keescook@...omium.org" <keescook@...omium.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "hjl.tools@...il.com" <hjl.tools@...il.com>,
        "x86@...nel.org" <x86@...nel.org>
Subject: Re: CET shadow stack app compatibility

On Tue, 2022-11-15 at 08:33 +0100, Florian Weimer wrote:
> Based on what we have seen so far (and since fixed), it's mostly
> shared
> objects that weren't marked up correctly.

For the benefit of anyone that is not involved in CET... As PeterZ was
just discussing, "CET" consists of two mostly independent features:
"IBT" and "Shadow Stack". I am currently trying to enable userspace
shadow stack in the kernel. No IBT enforcement will happen in userspace
for the time being.

For IBT, which seems to be in worse shape than shadow stack from an
existing userspace perspective, I have also seen shared objects with
issues.

For shadow stack, it was just JITing binaries. Of course if glibc is
compiled in non-permissive mode there is an additional category of
issues around dlopen()ing that we haven't even discussed yet. And the
past issues around makecontext() we have already worked around from the
kernel. If you are aware of any other specific compatibility problems,
please share so we can discuss the extent.

>   The posted hack didn't even
> deal with that case.  If the main executable has the current markers,
> the kernel will not disable shadow stack, and the process will still
> crash after loading the incorrectly marked shared object.

The proposed glibc changes would not enable shadow stack unless the
execing binary has the elf bit marked. So if we block those binaries
(which the kernel can easily check) from enabling shadow stack, none of
the linked shared objects will have shadow stack either. So I think we
are ok to hold this in our back pocket to resolve the known issues if
anyone complains.

Where the shared objects could come into play is, in the event that we
have to block the old elf bit from the kernel, and a new one is
properly marked on a new executable, future glibcs could decide to
honor the old bits when checking shared libraries. So you could have an
executable with SHSTK2 bit loading a problem SO with just SHSTK1 bit.

It would indeed be more difficult for the kernel to detect this,
especially in the dlopen() case, but it should not prevent simply
blocking any day 1 kernel support binaries. Please, please, don't do
this in the future if it comes up though. If the kernel can't find any
good options, it risks shadow stack getting reverted for everyone.

>   Someone has
> to step in and fix things for real (so that they don't break again
> just
> after rebuild with a current toolchain adding the current markers). 
> The
> kernel patch makes this harder because it's not possible anymore to
> use
> an existing distribution for this kind of work.

There was an EXPERT config for things like this, and I was mulling a
runtime sysctl. But I think now the idea is that the patch could serve
a "better than a full revert" purpose. Not an ideal solution.

But I still don't see why doing the order:
1. kernel support
2. libc support
3. compiler support

...wouldn't have generated a more normal situation where old binaries
don't break against new kernels and testing can easily happen to reduce
issues further. So we could still reset and do exactly that.

>   Instead, we'd have to
> wait for a rebuild with the new markers, and of course this rebuild
> will
> put is in exactly the same position as before: the incompatibilities
> will be back because they are no longer masked by the kernel.

People building new apps and testing them against upstream kernels and
finding issues sounds like business as usual. I'm not trying to solve
all possible userspace mistakes from the kernel.

Powered by blists - more mailing lists