linux-kernel - Re: [PATCH] x86/irq: Preserve vector in orig

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CALCETrX-8a61k03+XJop=k11-TkE+7JOiGTH=81sHXPmXsA+Tw@mail.gmail.com>
Date:   Wed, 26 Aug 2020 09:13:49 -0700
From:   Andy Lutomirski <luto@...nel.org>
To:     Thomas Gleixner <tglx@...utronix.de>
Cc:     Alexander Graf <graf@...zon.com>, X86 ML <x86@...nel.org>,
        Andy Lutomirski <luto@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Andrew Cooper <andrew.cooper3@...rix.com>,
        "Paul E. McKenney" <paulmck@...nel.org>,
        Alexandre Chartre <alexandre.chartre@...cle.com>,
        Frederic Weisbecker <frederic@...nel.org>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Sean Christopherson <sean.j.christopherson@...el.com>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Petr Mladek <pmladek@...e.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Joel Fernandes <joel@...lfernandes.org>,
        Boris Ostrovsky <boris.ostrovsky@...cle.com>,
        Juergen Gross <jgross@...e.com>,
        Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Will Deacon <will@...nel.org>,
        Tom Lendacky <thomas.lendacky@....com>,
        Wei Liu <wei.liu@...nel.org>,
        Michael Kelley <mikelley@...rosoft.com>,
        Jason Chen CJ <jason.cj.chen@...el.com>,
        Zhao Yakui <yakui.zhao@...el.com>,
        "Peter Zijlstra (Intel)" <peterz@...radead.org>,
        Avi Kivity <avi@...lladb.com>,
        "Herrenschmidt, Benjamin" <benh@...zon.com>, robketr@...zon.de,
        Amos Kong <amos@...lladb.com>, Brian Gerst <brgerst@...il.com>,
        stable <stable@...r.kernel.org>
Subject: Re: [PATCH] x86/irq: Preserve vector in orig_ax for APIC code

On Wed, Aug 26, 2020 at 7:27 AM Thomas Gleixner <tglx@...utronix.de> wrote:
>
> On Wed, Aug 26 2020 at 13:53, Alexander Graf wrote:
> > Commit 633260fa143 ("x86/irq: Convey vector as argument and not in ptregs")
> > changed the handover logic of the vector identifier from ~vector in orig_ax
> > to purely register based. Unfortunately, this field has another consumer
> > in the APIC code which the commit did not touch. The net result was that
> > IRQ balancing did not work and instead resulted in interrupt storms, slowing
> > down the system.
>
> The net result is an observationof the symptom but that does not explain
> what the underlying technical issue is.
>
> > This patch restores the original semantics that orig_ax contains the vector.
> > When we receive an interrupt now, the actual vector number stays stored in
> > the orig_ax field which then gets consumed by the APIC code.
> >
> > To ensure that nobody else trips over this in the future, the patch also adds
> > comments at strategic places to warn anyone who would refactor the code that
> > there is another consumer of the field.
> >
> > With this patch in place, IRQ balancing works as expected and performance
> > levels are restored to previous levels.
>
> There's a lot of 'This patch and we' in that changelog. Care to grep
> for 'This patch' in Documentation/process/ ?
>
> > diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
> > index df8c017..22e829c 100644
> > --- a/arch/x86/entry/entry_32.S
> > +++ b/arch/x86/entry/entry_32.S
> > @@ -727,7 +727,7 @@ SYM_CODE_START_LOCAL(asm_\cfunc)
> >       ENCODE_FRAME_POINTER
> >       movl    %esp, %eax
> >       movl    PT_ORIG_EAX(%esp), %edx         /* get the vector from stack */
> > -     movl    $-1, PT_ORIG_EAX(%esp)          /* no syscall to restart */
> > +     /* keep vector on stack for APIC's irq_complete_move() */
>
> Yes that's fixing your observed wreckage, but it introduces a worse one.
>
> user space
>   -> interrupt
>        push vector into orig_ax (values are in the ranges of 0-127 and -128 - 255
>                                  except for the system vectors which do
>                                  not go through this code)
>       handle()
>       ...
>       exit_to_user_mode_loop()
>          arch_do_signal()
>             /* Did we come from a system call? */
>             if (syscall_get_nr(current, regs) >= 0) {
>
>                ---> BOOM for any vector 0-127 because syscall_get_nr()
>                          resolves to regs->orig_ax
>
> Going to be fun to debug.
>
> The below nasty hack cures it, but I hate it with a passion. I'll look
> deeper for a sane variant.
>

Fundamentally, the way we overload orig_ax is problematic.  I have a
half-written series to improve it, but my series is broken.  I think
it's fixable, though.

First is this patch to use some __csh bits to indicate the entry type.
As far as I know, this patch is correct:

https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/entry&id=dfff54208072a27909ae97ebce644c251a233ff2

Then I wrote this incorrect patch:

https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/entry&id=3a5087acb8a2cc1e88b1a55fa36c2f8bef370572

That one is wrong because the orig_ax wreckage seems to have leaked
into user ABI -- user programs think that orig_ax has certain
semantics on user-visible entries.

But I think that the problem in this thread could be fixed quite
nicely by the first patch, plus a new CS_ENTRY_IRQ and allocating
eight bits of __csh to store the vector.  Then we could read out the
vector.


--Andy