linux-kernel - Re: [PATCH] kvm: add proper frame pointer logic for vmx

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9f1a99a9-a3c0-be89-8501-966e8f45ad8d@lca.pw>
Date:   Tue, 15 Jan 2019 11:54:38 -0500
From:   Qian Cai <cai@....pw>
To:     Sean Christopherson <sean.j.christopherson@...el.com>,
        Paolo Bonzini <pbonzini@...hat.com>
Cc:     rkrcmar@...hat.com, tglx@...utronix.de, mingo@...hat.com,
        bp@...en8.de, hpa@...or.com, x86@...nel.org, kvm@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] kvm: add proper frame pointer logic for vmx



On 1/15/19 11:34 AM, Sean Christopherson wrote:
> On Tue, Jan 15, 2019 at 08:13:22AM +0100, Paolo Bonzini wrote:
>> On 15/01/19 08:04, Qian Cai wrote:
>>>
>>>
>>> On 1/15/19 1:44 AM, Qian Cai wrote:
>>>> compilation warning since v5.0-rc1,
>>>>
>>>> arch/x86/kvm/vmx/vmx.o: warning: objtool: vmx_vcpu_run.part.17()+0x3171:
>>>> call without frame pointer save/setup
> 
> The warning is complaining about vmx_vcpu_run() in vmx.c, not vmenter.S.
> The rule being "broken" is that a call is made without creating a stack
> frame, and vmx_vmenter() obviously makes no calls.
> 
> E.g., manually running objtool check:
> 
>     $ tools/objtool/objtool check arch/x86/kvm/vmx/vmenter.o
>     $ tools/objtool/objtool check arch/x86/kvm/vmx/vmx.o
>     arch/x86/kvm/vmx/vmx.o: warning: objtool: vmx_vcpu_run.part.19()+0x83e: call without frame pointer save/setup
> 
> I put "broken" in quotes because AFAICT we're not actually violating the
> rule.  From tools/objtool/Documentation/stack-validation.txt:
> 
>    If it's a GCC-compiled .c file, the error may be because the function
>    uses an inline asm() statement which has a "call" instruction.  An
>    asm() statement with a call instruction must declare the use of the
>    stack pointer in its output operand.  On x86_64, this means adding
>    the ASM_CALL_CONSTRAINT as an output constraint:
> 
>      asm volatile("call func" : ASM_CALL_CONSTRAINT);
> 
>    Otherwise the stack frame may not get created before the call.
> 
> 
> The asm() blob that calls vmx_vmenter() uses ASM_CALL_CONSTRAINT, and
> the resulting asm output generates a frame pointer, e.g. this is from
> the vmx.o that objtool warns on:
> 
> Dump of assembler code for function vmx_vcpu_run:
>    0x0000000000007440 <+0>:     e8 00 00 00 00  callq  0x7445 <vmx_vcpu_run+5>
>    0x0000000000007445 <+5>:     55      push   %rbp
>    0x0000000000007446 <+6>:     48 89 e5        mov    %rsp,%rbp
> 
> 
> The warning only shows up in certain configs, e.g. I was only able to
> reproduce this using the .config provided by lkp.  Even explicitly
> enabling CONFIG_FRAME_POINTERS and CONFIG_STACK_VALIDATION didn't
> trigger the warning using my usual config.
> 
> And all that being said, I'm pretty sure this isn't related to the call
> to vmx_vmenter() at all, but rather is something that was exposed by
> removing __noclone from vmx_vcpu_run().
> 
> E.g. I still get the warning if I comment out the call to vmx_vmenter,
> it just shifts to something else (and continues to shift I comment out
> more calls).  The warning goes away if I re-add __noclone, regardless
> of whether or not commit 2bcbd406715d ("Revert "compiler-gcc: disable
> -ftracer for __noclone functions"") is applied.

It complained the call right here at the end of vmx_vcpu_run().

"callq 19eb6" in __read_once_size() via atomic_read()

and then jump back to vmx_vcpu_run() again.

/root/linux-debug/arch/x86/kvm/vmx/vmx.c:6650
}
   19e94:       e8 00 00 00 00          callq  19e99 <vmx_vcpu_run.part.21+0x3159>

__read_once_size():
/root/linux-debug/./include/linux/compiler.h:191
   19e99:       48 c7 c7 00 00 00 00    mov    $0x0,%rdi
   19ea0:       e8 00 00 00 00          callq  19ea5 <vmx_vcpu_run.part.21+0x3165>
   19ea5:       e9 8b dd ff ff          jmpq   17c35 <vmx_vcpu_run.part.21+0xef5>
   19eaa:       48 8b bd 48 ff ff ff    mov    -0xb8(%rbp),%rdi
   19eb1:       e8 00 00 00 00          callq  19eb6 <vmx_vcpu_run.part.21+0x3176>
   19eb6:       e9 b8 df ff ff          jmpq   17e73 <vmx_vcpu_run.part.21+0x1133>

vmx_vcpu_run():
/root/linux-debug/arch/x86/kvm/vmx/vmx.c:6621
        vcpu->arch.regs_dirty = 0;
   19ebb:       48 89 f7                mov    %rsi,%rdi
   19ebe:       e8 00 00 00 00          callq  19ec3 <vmx_vcpu_run.part.21+0x3183>
   19ec3:       e9 f1 e0 ff ff          jmpq   17fb9 <vmx_vcpu_run.part.21+0x1279>