linux-kernel - [PATCH v3 00/16] x86: Rewrite the retpoline rewrite logic

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20211026120132.613201817@infradead.org>
Date:   Tue, 26 Oct 2021 14:01:32 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     x86@...nel.org, jpoimboe@...hat.com, andrew.cooper3@...rix.com
Cc:     linux-kernel@...r.kernel.org, peterz@...radead.org,
        alexei.starovoitov@...il.com, ndesaulniers@...gle.com,
        bpf@...r.kernel.org
Subject: [PATCH v3 00/16] x86: Rewrite the retpoline rewrite logic

Hi,

These patches rewrite the way retpolines are rewritten. Currently objtool emits
alternative entries for most retpoline calls. However trying to extend that led
to trouble (ELF files are horrid).

Therefore completely overhaul this and have objtool emit a .retpoline_sites
section that lists all compiler generated retpoline thunk calls. Then the
kernel can do with them as it pleases.

Notably it will:

 - rewrite them to indirect instructions for !RETPOLINE
 - rewrite them to lfence; indirect; for RETPOLINE_AMD,
   where size allows (boo clang!)

Specifically, the !RETPOLINE case can now also deal with the clang-special
conditional-indirect-tail-call:

  Jcc __x86_indirect_thunk_\reg.

Finally, also update the x86 BPF jit to catch up to recent times and do these
same things.

All this should help improve performance by removing an indirection.

Patches can (soon) be found here:

  git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git objtool/core

Changes since v2:

 - rewrite the __x86_indirect_thunk_array[] stuff again
 - rewrite the retpoline,amd rewrite logic, it now also supports
   rewriting the Jcc case, if the original instruction is long enough, but
   more importantly, it's simpler code.
 - bpf label simplification patch
 - random assorted cleanups
 - actually managed to get bpf selftests working

---
 arch/um/kernel/um_arch.c                |   4 +
 arch/x86/include/asm/GEN-for-each-reg.h |  14 ++-
 arch/x86/include/asm/alternative.h      |   1 +
 arch/x86/include/asm/asm-prototypes.h   |  18 ---
 arch/x86/include/asm/nospec-branch.h    |  72 ++---------
 arch/x86/kernel/alternative.c           | 189 ++++++++++++++++++++++++++++-
 arch/x86/kernel/cpu/bugs.c              |   7 --
 arch/x86/kernel/module.c                |   9 +-
 arch/x86/kernel/vmlinux.lds.S           |  14 +++
 arch/x86/lib/retpoline.S                |  56 ++-------
 arch/x86/net/bpf_jit_comp.c             | 160 +++++++++---------------
 arch/x86/net/bpf_jit_comp32.c           |  22 +++-
 tools/objtool/arch/x86/decode.c         | 120 ------------------
 tools/objtool/check.c                   | 208 ++++++++++++++++++++++----------
 tools/objtool/elf.c                     |  84 -------------
 tools/objtool/include/objtool/check.h   |   1 -
 tools/objtool/include/objtool/elf.h     |   6 +-
 tools/objtool/special.c                 |   8 --
 18 files changed, 472 insertions(+), 521 deletions(-)