lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 04 Feb 2021 21:49:03 +0100
From:   Thomas Gleixner <>
To:     LKML <>
Cc:, Josh Poimboeuf <>,
        Kees Cook <>
Subject: [patch 00/12] x86/irq/64: Inline irq stack switching

The recent effort to make the ASM entry code slim and unified moved
the irq stack switching out of the low level ASM code so that the
whole return from interrupt work and state handling can be done in C
and the ASM code just handles the true low level details of entry and
exit (which is horrible enough already due to the well thought out

The main goal at this point was to get instrumentation and RCU state
under control in a validated way. Inlining the switch mechanism was
attempted back then, but that caused more objtool and unwinder trouble
than we had already on our plate, so we ended up with a simple,
functional but suboptimal implementation. The main issues are:

  - The unnecessary indirect call which is expensive thanks to
  - The inability to stay on the irq stack for softirq processing on return
    from interrupt which requires another stack switch operation.
  - The fact that the stack switching code ended up being an easy to find
    exploit gadget.

This series revisits the problem and reimplements the stack switch
mechanics via evil inline assembly. Peter Zijlstra provided the required
objtool and unwinder changes already. These are available here:

The full series (including Peter's series) is also available from git:

  git:// x86/entry

All function calls are now direct and fully inlined including the single
instance in the softirq code which is invoked from local_bh_enable() in
task context.

The extra 100 lines in the diffstat are pretty much the extensive commentry
for the whole magic to spare everyone including myself to scratch heads 2
weeks down the road. 

The text size impact is in the noise and looking at the actual entry
functions there is depending on the compiler variant even a small size

The patches have been tested with gcc8, gcc10 and clang-13 (fresh from
git). The difference between the output of these compilers is minimal.
gcc8 being slightly worse due to stupid register selection and random
NOPs injected.


 arch/x86/Kconfig                 |    1 
 arch/x86/entry/common.c          |   19 --
 arch/x86/entry/entry_64.S        |   41 -----
 arch/x86/include/asm/idtentry.h  |   11 -
 arch/x86/include/asm/irq.h       |    3 
 arch/x86/include/asm/irq_stack.h |  283 +++++++++++++++++++++++++++------------
 arch/x86/include/asm/processor.h |    9 -
 arch/x86/kernel/apic/apic.c      |   31 ++--
 arch/x86/kernel/cpu/common.c     |    4 
 arch/x86/kernel/dumpstack_64.c   |   22 ++-
 arch/x86/kernel/irq.c            |    2 
 arch/x86/kernel/irq_64.c         |   11 -
 arch/x86/kernel/process_64.c     |    2 
 include/linux/interrupt.h        |    2 
 kernel/softirq.c                 |    4 
 15 files changed, 270 insertions(+), 175 deletions(-)

Powered by blists - more mailing lists