| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAG48ez1rkN0YU-ieBaUZDKFYG5XFnd7dhDjSDdRmVfWyQzsA5g@mail.gmail.com>
Date: Sun, 1 Mar 2020 07:02:15 +0100
From: Jann Horn <jannh@...gle.com>
To: Andy Lutomirski <luto@...nel.org>,
Thomas Gleixner <tglx@...utronix.de>,
Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
"H. Peter Anvin" <hpa@...or.com>,
"the arch/x86 maintainers" <x86@...nel.org>
Cc: kernel list <linux-kernel@...r.kernel.org>,
Josh Poimboeuf <jpoimboe@...hat.com>
Subject: x86 entry perf unwinding failure (missing IRET_REGS annotation on
stack switch?)
Hi!
With the latest kernel from Linus' tree (fb279f4e2386), on a box that
needs KPTI, as soon as I run "sudo <path>/perf top -g", I see this
warning (if I modify the unwinding code to use %px there instead of
%p):
[ 83.716065] WARNING: stack recursion on stack type 4
[ 83.716070] WARNING: can't dereference registers at
fffffe0000002190 for ip
swapgs_restore_regs_and_return_to_usermode+0x79/0x80
So apparently some ORC annotations are a bit off. I decided to write a
little helper script that can interleave "objtool orc dump" and
"objdump -d" output to make it easier for me to understand what's
going on; you can save it to tools/objtool and run it from there:
==============
#!/usr/bin/env python3
import subprocess
import sys
import re
objfile = sys.argv[1]
orc_dump = subprocess.run([sys.path[0]+'/objtool', 'orc', 'dump',
objfile], stdout=subprocess.PIPE).stdout.decode('utf-8')
orc_map = {}
for line in orc_dump.split('\n'):
parts = line.split(': ', 1)
if len(parts) != 2:
continue
orc_map[parts[0]] = parts[1]
objdump = subprocess.run(['objdump', '-d', objfile],
stdout=subprocess.PIPE).stdout.decode('utf-8')
cur_section = None
hex_re = re.compile('^[a-f0-9]+$')
for line in objdump.split('\n'):
if line.startswith('Disassembly of section '):
cur_section = line.split('of section ')[1].split(':')[0]
if line.find(':') != -1:
hex_prefix = line.split(':')[0].lstrip()
if hex_re.match(hex_prefix):
orc_val = orc_map.get(cur_section+'+'+hex_prefix)
if orc_val != None:
print('\033[92m#######' + orc_val + '\033[0m')
print(line)
==============
swapgs_restore_regs_and_return_to_usermode+0x79/0x80 is offset 0xaa8
in the object file. Here's the interleaved assembly and ORC unwind
info (with ORC info appearing above the line it refers to):
==============
0000000000000a00 <common_interrupt>:
#######sp:sp+8 bp:(und) type:iret end:0
a00: 48 83 04 24 80 addq $0xffffffffffffff80,(%rsp)
a05: e8 00 00 00 00 callq a0a <common_interrupt+0xa>
#######sp:(sp+0) bp:(und) type:regs end:0
a0a: e8 00 00 00 00 callq a0f <ret_from_intr>
0000000000000a0f <ret_from_intr>:
a0f: fa cli
a10: 5c pop %rsp
#######sp:sp+0 bp:(und) type:regs end:0
a11: 65 ff 0c 25 00 00 00 decl %gs:0x0
a18: 00
a19: f6 84 24 88 00 00 00 testb $0x3,0x88(%rsp)
a20: 03
a21: 0f 84 88 00 00 00 je aaf <retint_kernel>
a27: 48 89 e7 mov %rsp,%rdi
a2a: e8 00 00 00 00 callq a2f
<swapgs_restore_regs_and_return_to_usermode>
0000000000000a2f <swapgs_restore_regs_and_return_to_usermode>:
a2f: 41 5f pop %r15
#######sp:sp-8 bp:(und) type:regs end:0
a31: 41 5e pop %r14
#######sp:sp-16 bp:(und) type:regs end:0
a33: 41 5d pop %r13
#######sp:sp-24 bp:(und) type:regs end:0
a35: 41 5c pop %r12
#######sp:sp-32 bp:(und) type:regs end:0
a37: 5d pop %rbp
#######sp:sp-40 bp:(und) type:regs end:0
a38: 5b pop %rbx
#######sp:sp-48 bp:(und) type:regs end:0
a39: 41 5b pop %r11
#######sp:sp-56 bp:(und) type:regs end:0
a3b: 41 5a pop %r10
#######sp:sp-64 bp:(und) type:regs end:0
a3d: 41 59 pop %r9
#######sp:sp-72 bp:(und) type:regs end:0
a3f: 41 58 pop %r8
#######sp:sp-80 bp:(und) type:regs end:0
a41: 58 pop %rax
#######sp:sp-88 bp:(und) type:regs end:0
a42: 59 pop %rcx
#######sp:sp-96 bp:(und) type:regs end:0
a43: 5a pop %rdx
#######sp:sp-104 bp:(und) type:regs end:0
a44: 5e pop %rsi
#######sp:sp-112 bp:(und) type:regs end:0
a45: 48 89 e7 mov %rsp,%rdi
a48: 65 48 8b 24 25 00 00 mov %gs:0x0,%rsp
a4f: 00 00
a51: ff 77 30 pushq 0x30(%rdi)
#######sp:sp-104 bp:(und) type:regs end:0
a54: ff 77 28 pushq 0x28(%rdi)
#######sp:sp-96 bp:(und) type:regs end:0
a57: ff 77 20 pushq 0x20(%rdi)
#######sp:sp-88 bp:(und) type:regs end:0
a5a: ff 77 18 pushq 0x18(%rdi)
#######sp:sp-80 bp:(und) type:regs end:0
a5d: ff 77 10 pushq 0x10(%rdi)
#######sp:sp-72 bp:(und) type:regs end:0
a60: ff 37 pushq (%rdi)
#######sp:sp-64 bp:(und) type:regs end:0
a62: 50 push %rax
#######sp:sp-56 bp:(und) type:regs end:0
a63: eb 43 jmp aa8
<swapgs_restore_regs_and_return_to_usermode+0x79>
a65: 0f 20 df mov %cr3,%rdi
a68: eb 34 jmp a9e
<swapgs_restore_regs_and_return_to_usermode+0x6f>
a6a: 48 89 f8 mov %rdi,%rax
a6d: 48 81 e7 ff 07 00 00 and $0x7ff,%rdi
a74: 65 48 0f a3 3c 25 00 bt %rdi,%gs:0x0
a7b: 00 00 00
a7e: 73 0f jae a8f
<swapgs_restore_regs_and_return_to_usermode+0x60>
a80: 65 48 0f b3 3c 25 00 btr %rdi,%gs:0x0
a87: 00 00 00
a8a: 48 89 c7 mov %rax,%rdi
a8d: eb 08 jmp a97
<swapgs_restore_regs_and_return_to_usermode+0x68>
a8f: 48 89 c7 mov %rax,%rdi
a92: 48 0f ba ef 3f bts $0x3f,%rdi
a97: 48 81 cf 00 08 00 00 or $0x800,%rdi
a9e: 48 81 cf 00 10 00 00 or $0x1000,%rdi
aa5: 0f 22 df mov %rdi,%cr3
aa8: 58 pop %rax
#######sp:sp-64 bp:(und) type:regs end:0
aa9: 5f pop %rdi
#######sp:sp-72 bp:(und) type:regs end:0
aaa: 0f 01 f8 swapgs
aad: eb 41 jmp af0 <native_iret>
==============
It looks to me like things go wrong at the point where we switch over
to the trampoline stack? The ORC info claims that we have full user
registers on the trampoline stack (and that we're clobbering them with
our pushes - apparently objtool is not smart enough to realize that
that looks bogus), but at that point we should probably actually use
something like UNWIND_HINT_IRET_REGS, right?
By the way, looking through the rest of the entry stuff, there's some
other funny-looking stuff, too:
============
0000000000000f40 <general_protection>:
#######sp:sp+8 bp:(und) type:iret end:0
f40: 90 nop
#######sp:(und) bp:(und) type:call end:0
f41: 90 nop
f42: 90 nop
#######sp:sp+8 bp:(und) type:iret end:0
f43: e8 a8 01 00 00 callq 10f0 <error_entry>
#######sp:sp+0 bp:(und) type:regs end:0
f48: f6 84 24 88 00 00 00 testb $0x3,0x88(%rsp)
f4f: 03
f50: 74 00 je f52 <general_protection+0x12>
f52: 48 89 e7 mov %rsp,%rdi
f55: 48 8b 74 24 78 mov 0x78(%rsp),%rsi
f5a: 48 c7 44 24 78 ff ff movq $0xffffffffffffffff,0x78(%rsp)
f61: ff ff
f63: e8 00 00 00 00 callq f68 <general_protection+0x28>
f68: e9 73 02 00 00 jmpq 11e0 <error_exit>
#######sp:(und) bp:(und) type:call end:0
f6d: 0f 1f 00 nopl (%rax)
============
So I think on machines without X86_FEATURE_SMAP, trying to unwind from
the two NOPs at f41 and f42 will cause the unwinder to report an
error? Looking at unwind_next_frame(), "sp:(und)" without the "end:1"
marker seems to be reserved for errors.
Powered by blists - more mailing lists