linux-kernel - Re: [RFC PATCH] x86: optimize IRET returns to kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 31 Mar 2015 09:49:31 -0400
From:	Steven Rostedt <rostedt@...dmis.org>
To:	Denys Vlasenko <dvlasenk@...hat.com>
Cc:	Andy Lutomirski <luto@...capital.net>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Ingo Molnar <mingo@...nel.org>, Borislav Petkov <bp@...en8.de>,
	"H. Peter Anvin" <hpa@...or.com>, Oleg Nesterov <oleg@...hat.com>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Alexei Starovoitov <ast@...mgrid.com>,
	Will Drewry <wad@...omium.org>,
	Kees Cook <keescook@...omium.org>, x86@...nel.org,
	linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH] x86: optimize IRET returns to kernel

On Tue, 31 Mar 2015 14:46:21 +0200
Denys Vlasenko <dvlasenk@...hat.com> wrote:

> @@ -750,6 +750,53 @@ retint_kernel:
>  	 * The iretq could re-enable interrupts:
>  	 */
>  	TRACE_IRQS_IRETQ
> +
> +	/*
> +	 * Since we return to kernel, CS and SS do not need changing.
> +	 * Only RSP, RIP and RFLAGS do.
> +	 * We can use POPF + near RET, which is much faster.
> +	 * The below code may seem excessive, but IRET is _very_ slow.
> +	 * Hundreds of cycles.
> +	 *
> +	 * However, there is a complication. Interrupts in 64-bit mode
> +	 * align stack to 16 bytes. This changes location
> +	 * where we need to store EFLAGS and RIP:
> +	 */
> +#if 0
> +	testb	$8, RSP(%rsp)

Shouldn't this be: testb $0xf, RSP(%rsp) ?

The stack should be word (8 bytes) aligned, but I never like to assume
anything. As an interrupt can come in anywhere, and %rsp can be
modified to anything in assembly, I wouldn't want some "hack" that
performs a non word aligned rsp manipulation to suddenly break. It
would be rather hard to debug.


> +	jnz	1f
> +#else
> +	/* There is a complication #2: 64-bit mode has IST stacks */
> +	leaq	SIZEOF_PTREGS+8(%rsp), %rax
> +	cmpq	%rax, RSP(%rsp)
> +	je	1f
> +	subq	$8, %rax
> +	cmpq	%rax, RSP(%rsp)
> +	jne	restore_args /* probably IST stack, can't optimize */
> +#endif
> +	/* there is no padding above iret frame */
> +	movq	EFLAGS(%rsp), %rax
> +	movq	RIP(%rsp), %rcx
> +	movq	%rax, (SIZEOF_PTREGS-2*8)(%rsp)
> +	movq	%rcx, (SIZEOF_PTREGS-1*8)(%rsp)
> +	CFI_REMEMBER_STATE
> +	RESTORE_C_REGS
> +	REMOVE_PT_GPREGS_FROM_STACK 4*8 /* remove all except last two words */
> +	popfq_cfi
> +	retq

BTW, have you made sure that this path has been hit?

-- Steve

> +	CFI_RESTORE_STATE
> +1:	/* there are 8 bytes of padding above iret frame */
> +	movq	EFLAGS(%rsp), %rax
> +	movq	RIP(%rsp), %rcx
> +	movq	%rax, (SIZEOF_PTREGS-2*8 + 8)(%rsp)
> +	movq	%rcx, (SIZEOF_PTREGS-1*8 + 8)(%rsp)
> +	CFI_REMEMBER_STATE
> +	RESTORE_C_REGS
> +	REMOVE_PT_GPREGS_FROM_STACK 4*8 + 8
> +	popfq_cfi
> +	retq
> +	CFI_RESTORE_STATE
> +
>  restore_args:
>  	RESTORE_C_REGS
>  	REMOVE_PT_GPREGS_FROM_STACK 8

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/