linux-kernel - Re: [RFC PATCH v6 1/5] Thread-local ABI system call: cache CPU number of running thread

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <966747921.48958.1460044750563.JavaMail.zimbra@efficios.com>
Date:	Thu, 7 Apr 2016 15:59:10 +0000 (UTC)
From:	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Florian Weimer <fweimer@...hat.com>,
	"H. Peter Anvin" <hpa@...or.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Russell King <linux@....linux.org.uk>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...hat.com>, linux-kernel@...r.kernel.org,
	linux-api <linux-api@...r.kernel.org>,
	Paul Turner <pjt@...gle.com>, Andrew Hunter <ahh@...gle.com>,
	Andy Lutomirski <luto@...capital.net>,
	Andi Kleen <andi@...stfloor.org>,
	Dave Watson <davejwatson@...com>, Chris Lameter <cl@...ux.com>,
	Ben Maurer <bmaurer@...com>, rostedt <rostedt@...dmis.org>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Josh Triplett <josh@...htriplett.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Catalin Marinas <catalin.marinas@....com>,
	Will Deacon <will.deacon@....com>,
	Michael Kerrisk <mtk.manpages@...il.com>,
	Boqun Feng <boqun.feng@...il.com>
Subject: Re: [RFC PATCH v6 1/5] Thread-local ABI system call: cache CPU
 number of running thread

----- On Apr 7, 2016, at 8:25 AM, Peter Zijlstra peterz@...radead.org wrote:

> On Thu, Apr 07, 2016 at 02:03:53PM +0200, Florian Weimer wrote:
>> > struct tlabi {
>> > 	union {
>> > 		__u8[64] __foo;
>> > 		struct {
>> > 			/* fields go here */
>> > 		};
>> > 	};
>> > } __aligned__(64);
>> 
>> That's not really “fixed size” as far as an ABI is concerned, due to the
>> possibility of future extensions.
> 
> sizeof(struct tlabi) is always the same, right? How is that not fixed?
> 
>> > People objected against the fixed size scheme, but it being possible to
>> > get a fixed TCB offset and reduce indirections is a big win IMO.
>> 
>> It's a difficult trade-off.  It's not an indirection as such, it's avoid
>> loading the dynamic TLS offset.
> 
> What we _want_ is being able to use %[gf]s:offset and have it work (I
> forever forget which segment register userspace TLS uses).
> 
>> Let me repeat that the ELF TLS GNU ABI has very limited support for
>> static offsets at present, and it is difficult to make them available
>> more widely without code generation at run time (in the form of text
>> relocations, but still).
> 
> Do you have a pointer to something I can read? Because I'm clearly not
> understanding the full issue here.

For what is is worth, here are a couple of objdump snippet of my
test program without and with -fPIC:

* Compiled with -O2, *without* -fPIC, x86-64:

__thread __attribute__((weak)) volatile struct thread_local_abi __thread_local_abi;

static
int32_t read_cpu_id(void)
{
        if (unlikely(!(__thread_local_abi.features & TLABI_FEATURE_CPU_ID)))
  40064e:       64 8b 04 25 c0 ff ff    mov    %fs:0xffffffffffffffc0,%eax
  400655:       ff 
  400656:       a8 01                   test   $0x1,%al
  400658:       74 71                   je     4006cb <main+0xab>
                return sched_getcpu();
        return __thread_local_abi.cpu_id;
  40065a:       64 8b 14 25 c4 ff ff    mov    %fs:0xffffffffffffffc4,%edx
  400661:       ff 
}


* Compiled with -O2, with -fPIC, x86_64:

__thread __attribute__((weak)) volatile struct thread_local_abi __thread_local_abi;


  4006de:       64 48 8b 04 25 00 00    mov    %fs:0x0,%rax
  4006e5:       00 00 

static
int32_t read_cpu_id(void)
{
        if (unlikely(!(__thread_local_abi.features & TLABI_FEATURE_CPU_ID)))
  4006e7:       48 8d 80 c0 ff ff ff    lea    -0x40(%rax),%rax
  4006ee:       8b 10                   mov    (%rax),%edx
  4006f0:       83 e2 01                and    $0x1,%edx
  4006f3:       0f 84 80 00 00 00       je     400779 <main+0xc9>
                return sched_getcpu();
        return __thread_local_abi.cpu_id;
  4006f9:       8b 50 04                mov    0x4(%rax),%edx
}

So with -fPIC (libraries), TLS adds an extra indirection. However,
it just needs to load the base address once, and can then access
both "features" and "cpu_id" fields as offsets from that base.

For executables compiled without -fPIC, there is no indirection.
This justifies the following paragraph in the proposed man page:

           The symbol __thread_local_abi is recommended  to  be  used  across
           libraries  and applications wishing to register a the thread-local
           ABI structure for tlabi_nr 0. The attribute "weak" is  recommended
           when  declaring  this  variable  in  libraries.   Applications can
           choose to define their own version of this symbol without the weak
           attribute as a performance improvement.

Thoughts ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com