linux-kernel - Re: [RFC PATCH v2] x86/arch_prctl: Add ARCH_SET

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200407123348.GV20730@hirez.programming.kicks-ass.net>
Date:   Tue, 7 Apr 2020 14:33:48 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Kyle Huey <me@...ehuey.com>
Cc:     Keno Fischer <keno@...iacomputing.com>,
        Andy Lutomirski <luto@...capital.net>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>,
        "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" <x86@...nel.org>,
        "H. Peter Anvin" <hpa@...or.com>, Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Andi Kleen <andi@...stfloor.org>,
        Kyle Huey <khuey@...ehuey.com>,
        Robert O'Callahan <robert@...llahan.org>
Subject: Re: [RFC PATCH v2] x86/arch_prctl: Add ARCH_SET_XCR0 to set XCR0
 per-thread

On Mon, Apr 06, 2020 at 09:53:40PM -0700, Kyle Huey wrote:
> On Mon, Apr 6, 2020 at 9:45 PM Keno Fischer <keno@...iacomputing.com> wrote:
> >
> > On Mon, Apr 6, 2020 at 11:58 PM Andy Lutomirski <luto@...capital.net> wrote:
> > >
> > >
> > > > On Apr 6, 2020, at 6:13 PM, Keno Fischer <keno@...iacomputing.com> wrote:
> > > >
> > > > This is a follow-up to my from two-years ago [1].
> > >
> > > Your changelog is missing an explanation of why this is useful.  Why would a user program want to change XCR0?
> >
> > Ah, sorry - I wasn't sure what the convention was around repeating the
> > applicable parts from the v1 changelog in this email.
> > Here's the description from the v1 patch:
> >
> > > The rr (http://rr-project.org/) debugger provides user space
> > > record-and-replay functionality by carefully controlling the process
> > > environment in order to ensure completely deterministic execution
> > > of recorded traces. The recently added ARCH_SET_CPUID arch_prctl
> > > allows rr to move traces across (Intel) machines, by allowing cpuid
> > > invocations to be reliably recorded and replayed. This works very
> > > well, with one catch: It is currently not possible to replay a
> > > recording from a machine supporting a smaller set of XCR0 state
> > > components on one supporting a larger set. This is because the
> > > value of XCR0 is observable in userspace (either by explicit
> > > xgetbv or by looking at the result of xsave) and since glibc
> > > does observe this value, replay divergence is almost immediate.
> > > I also suspect that people interested in process (or container)
> > > live-migration may eventually care about this if a migration happens
> > > in between a userspace xsave and a corresponding xrstor.
> > >
> > > We encounter this problem quite frequently since most of our users
> > > are using pre-Skylake systems (and thus don't support the AVX512
> > > state components), while we recently upgraded our main development
> > > machines to Skylake.
> >
> > Basically, for rr to work, we need to tightly control any user-visible
> > CPU behavior,
> > either by putting in the CPU in the right state or by trapping and emulating
> > (as we do for rdtsc, cpuid, etc). XCR0 controls a bunch of
> > user-visible CPU behavior,
> > namely:
> > 1) The size of the xsave region if xsave is passed an all-ones mask
> > (which is fairly common)
> > 2) The return value of xgetbv
> 
> It's mentioned elsewhere, but I want to emphasize that the return
> value of xgetbv is the big one because the dynamic linker uses this.
> rr trace portability is essentially limited to machines with identical
> xcr0 values because of it.

I'm thinking just exposing that value is doable in a much less
objectionable fashion, no?