Message-ID: <CAMe9rOo0E7bvB_SkTcd9bEhUoejRduDboDPy+c5FVVVufpc87w@mail.gmail.com>
Date:   Sat, 17 Jun 2017 12:48:27 -0700
From:   "H.J. Lu" <hjl.tools@...il.com>
To:     Andy Lutomirski <luto@...nel.org>
Cc:     Dave Hansen <dave.hansen@...el.com>,
        "Robert O'Callahan" <robert@...llahan.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        X86 ML <x86@...nel.org>
Subject: Re: xgetbv nondeterminism

On Sat, Jun 17, 2017 at 9:32 AM, Andy Lutomirski <luto@...nel.org> wrote:
> On Sat, Jun 17, 2017 at 5:51 AM, H.J. Lu <hjl.tools@...il.com> wrote:
>> On Fri, Jun 16, 2017 at 11:21 PM, Andy Lutomirski <luto@...nel.org> wrote:
>>>>>
>>>>> In any event, I still don't understand the issue.  The code does this,
>>>>> effectively:
>>>>>
>>>>> PLT -> GOT
>>>>> GOT points to a stub that transfers control to ld.so
>>>>> ld.so resolves the symbol (_dl_fixup, I think)
>>>>> ld.so patches the GOT
>>>>> ld.so jumps to the resolved function
>>>>>
>>>>> As far as I can tell, the only part of the whole process that might
>>>>> touch vector registers at all is elf_ifunc_invoke().  Couldn't all the
>>>>> register saving and restoring be moved to elf_ifunc_invoke()?
>>>>
>>>> Please grep for  FOREIGN_CALL the elf directory.
>>>
>>> I grepped FOREIGN_CALL.  It has no explanation whatsoever and appears
>>> to unconditionally do nothing in the current glibc version.
>>>
>>> In f3dcae82d54e5097e18e1d6ef4ff55c2ea4e621e^, in pseudocode, it does:
>>>
>>> __thread bool must_save;
>>>
>>> RTLD_CHECK_FOREIGN_CALL: return must_save;
>>>
>>> RTLD_ENABLE_FOREIGN_CALL: old_must_save = must_save; must_save = true;
>>>
>>> RTLD_PREPARE_FOREIGN_CALL: save_state(); must_save = false;
>>>
>>> RTLD_FINALIZE_FOREIGN_CALL: if (must_save) restore(); must_save = old_must_save;
>>>
>>> save_state() and restore_state() operate on TLS buffers.
>>>
>>> In summary: this is not async-signal-safe.  It's also really messy --
>>> there are macros that declare local variables, and the logic isn't
>>> apparent without really digging in to all the code.
>>>
>>> I still don't see why this couldn't be:
>>>
>>> static void elf_do_foreign_stuff(args here)
>>> {
>>>   void *buf = alloca(state_size);
>>>   xsaveopt(buf);  /* or open-code it if you prefer */
>>>   call_the_ifunc();
>>>   xrstor(buf);
>>> }
>>
>> As you have found out that it doesn't work this way since
>>
>> RTLD_PREPARE_FOREIGN_CALL
>>
>> and
>>
>> RTLD_FINALIZE_FOREIGN_CALL
>>
>> are used in 2 DIFFERENT files.
>>
>
> That's ought to be fixable, either by rearranging code or by doing
> something like:
>
> RTLD_INIT_FOREIGN_CALL(foreign_call_state);
>
> _dl_whatever_helper(&foreign_call_state);
>
> RTLD_FINALIZE_FOREIGN_CALL(foreign_call_state);
>
> _dl_whatever_helper would do
> RTLD_PREPARE_FOREIGN_CALL(ptr_to_foreign_call_state);
>
> renaming these macros a bit might help, too.

We have no plan to do anything.

>>> If there's more than just the iifunc (malloc?  profiling?  printf?)
>>> then all of that could be wrapped as well.
>>
>> It has nothing to do with ifunc.
>
> What's it for, then?  I don't understand why, in a sensible ld.so
> architecture, there would ever be a call out from ld.so during runtime
> binding to anything other than an ifunc, but I realize that glibc is
> weird and ld.so might call out to libc.so for some reason.  It doesn't
> really matter, though.
>
>

ld.so has a set of minimal implementations of some library function,
like malloc, free, ..., which are used by ld.so to bootstrap itself.  After
libc.so is loaded and relocated, the full implementations from libc.so
are used in ld.so.  Since they are outside of ld.so, they are foreign
calls to ld.so and ld.so needs to preserve the first 8 vector registers
when the foreign call is made to libc.so.

If there are any suggestions for ld.so, please discuss them at

https://sourceware.org/ml/libc-alpha/

Thanks.

-- 
H.J.

Powered by blists - more mailing lists