Date:	Mon, 17 Aug 2015 06:36:03 +0200
From:	Borislav Petkov <bp@...e.de>
To:	Jiri Olsa <jolsa@...hat.com>
Cc:	linux-kernel@...r.kernel.org, x86@...nel.org,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...e.de>,
	Robert Richter <rric@...nel.org>,
	"H. Peter Anvin" <hpa@...or.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Namhyung Kim <namhyung@...nel.org>,
	Jan Stancek <jstancek@...hat.com>,
	Andy Lutomirski <luto@...capital.net>
Subject: Re: [BUG/RFC] perf test fails on AMD CPUs

On Mon, Aug 17, 2015 at 12:29:56AM +0200, Jiri Olsa wrote:
> hi,
> 'perf test 18' is failing on systems with AMD processor.

Hmm, still using that b0rked test box? :-)

Also, which kernel?

There have been substantial changes to the entry code recently. Although
I don't see anything being done differently on AMD there except
X86_BUG_SYSRET_SS_ATTRS but that should be unrelated.

> The only reason I could find is that AMD does not set 'resume flag'
> in RFLAGS register the way the Intel CPU does.
> 
> (simplified) test scenario:
> 
>   - create breakpoint (on test_function) perf event with SIGIO signal
>     to be delivered any time the breakpoint is hit
>   - run test_function
>   
> 
> expected course of actions is:
>   1) CPU hits 'test_function'
>   2) DB exception is triggered, with RFLAGS.RF=0
>   3) DB exception handler sets regs->RFLAGS.RF=1 and perf handler
>      triggers irq_work pending work
>   4) DB exception executes iretd
>   5) irq_work interrupt is triggered, with RFLAGS.RF=1
>   6) irq_work interrupt calls kill_fasync with SIGIO signal
>   7) irq_work interrupt on return to userspace calls prepare_exit_to_usermode
>      which actually delivers the SIGIO signal
>   8) sigreturn syscall prepare registers to return to the
>      instruction from step 1) and sets RFLAGS.RF to the its original
>      value from step 5) (RFLAGS.RF=1)
>   9) CPU hits 'test_function' and DB exception is NOT triggered
>      due to RFLAGS.RF=1
> 
> this is how I see it works on Intel
> 
> But AMD gives me RFLAGS.RF=0 on step 5, which makes the step 9 to
> trigger the DB exception once again and makes the test fail.

Adding Andy, he might have an idea. Leaving in the rest for reference.

> I'm not sure this test ever worked on AMD CPUs, anyway is there
> anything I'm missing or is this some AMD/Intel quirk?
> 
> thanks,
> jirka
> 
> 
> 
> AMD description of RF flag (SDM 3.1.6):
> =======================================
> Resume Flag (RF) Bit. Bit 16. The RF bit allows an instruction to be restarted following an
> instruction breakpoint resulting in a debug exception (#DB). This bit prevents multiple debug
> exceptions from occurring on the same instruction.
> The processor clears the RF bit after every instruction is successfully executed, except when the
> instruction is:
> •
> •
> An IRET that sets the RF bit.
> JMP, CALL, or INTn through a task gate.
> In both of the above cases, RF is not cleared to 0 until the next instruction successfully executes.
> When an exception occurs (or when a string instruction is interrupted), the processor normally sets
> RF=1 in the RFLAGS image saved on the interrupt stack. However, when a #DB exception occurs as a
> result of an instruction breakpoint, the processor clears the RF bit to 0 in the interrupt-stack RFLAGS
> image.
> For instruction restart to work properly following an instruction breakpoint, the #DB exception
> handler must set RF to 1 in the interrupt-stack RFLAGS image. When an IRET is later executed to
> return to the instruction that caused the instruction-breakpoint #DB exception, the set RF bit (RF=1) is
> loaded from the interrupt-stack RFLAGS image. RF is not cleared by the processor until the
> instruction causing the #DB exception successfully executes.
> 
> Intel description of RF flag (SDM 17.3.1.1):
> ============================================
> Because the debug exception for an instruction breakpoint is generated before the instruction is executed, if the
> instruction breakpoint is not removed by the exception handler; the processor will detect the instruction breakpoint
> again when the instruction is restarted and generate another debug exception. To prevent looping on an instruction
> breakpoint, the Intel 64 and IA-32 architectures provide the RF flag (resume flag) in the EFLAGS register (see
> Section 2.3, “System Flags and Fields in the EFLAGS Register,” in the Intel® 64 and IA-32 Architectures Software
> Developer’s Manual, Volume 3A). When the RF flag is set, the processor ignores instruction breakpoints.
> All Intel 64 and IA-32 processors manage the RF flag as follows. The RF Flag is cleared at the start of the instruction
> after the check for code breakpoint, CS limit violation and FP exceptions. Task Switches and IRETD/IRETQ instruc-
> tions transfer the RF image from the TSS/stack to the EFLAGS register.
> When calling an event handler, Intel 64 and IA-32 processors establish the value of the RF flag in the EFLAGS image
> pushed on the stack:
> • For any fault-class exception except a debug exception generated in response to an instruction breakpoint, the
> value pushed for RF is 1.
> • For any interrupt arriving after any iteration of a repeated string instruction but the last iteration, the value
> pushed for RF is 1.
> • For any trap-class exception generated by any iteration of a repeated string instruction but the last iteration,
> the value pushed for RF is 1.
> • For other cases, the value pushed for RF is the value that was in EFLAG.RF at the time the event handler was
> called. This includes:
> — Debug exceptions generated in response to instruction breakpoints
> — Hardware-generated interrupts arriving between instructions (including those arriving after the last
> iteration of a repeated string instruction)
> — Trap-class exceptions generated after an instruction completes (including those generated after the last
> iteration of a repeated string instruction)
> — Software-generated interrupts (RF is pushed as 0, since it was cleared at the start of the software interrupt)
> As noted above, the processor does not set the RF flag prior to calling the debug exception handler for debug
> exceptions resulting from instruction breakpoints. The debug exception handler can prevent recurrence of the
> instruction breakpoint by setting the RF flag in the EFLAGS image on the stack. If the RF flag in the EFLAGS image
> 17-8 Vol. 3BDEBUG, BRANCH PROFILE, TSC, AND RESOURCE MONITORING FEATURES
> is set when the processor returns from the exception handler, it is copied into the RF flag in the EFLAGS register by
> IRETD/IRETQ or a task switch that causes the return. The processor then ignores instruction breakpoints for the
> duration of the next instruction. (Note that the POPF, POPFD, and IRET instructions do not transfer the RF image
> into the EFLAGS register.) Setting the RF flag does not prevent other types of debug-exception conditions (such as,
> I/O or data breakpoints) from being detected, nor does it prevent non-debug exceptions from being generated.
> For the Pentium processor, when an instruction breakpoint coincides with another fault-type exception (such as a
> page fault), the processor may generate one spurious debug exception after the second exception has been
> handled, even though the debug exception handler set the RF flag in the EFLAGS image. To prevent a spurious
> exception with Pentium processors, all fault-class exception handlers should set the RF flag in the EFLAGS image.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists