linux-kernel - Re: [PATCH 2/5] x86_64: inline copy

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170428210409.ap3bnb4i3l4gm36p@pd.tnic>
Date:   Fri, 28 Apr 2017 23:04:09 +0200
From:   Borislav Petkov <bp@...en8.de>
To:     Alexey Dobriyan <adobriyan@...il.com>
Cc:     x86@...nel.org, tglx@...utronix.de, mingo@...hat.com,
        hpa@...or.com, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 2/5] x86_64: inline copy_page() at call site

On Wed, Apr 26, 2017 at 09:28:06PM +0300, Alexey Dobriyan wrote:
> Avoid unconditional branch at every copy_page() call by using
> alternatives and calling optimal variant directly.
> 
> Rename individual versions to immediately show which one is used in
> profiles, etc.
> 
> RBX and R12 aren't clobbered because generic version restores them
> and both REP versions don't touch them.
> 
> Signed-off-by: Alexey Dobriyan <adobriyan@...il.com>
> ---
> 
>  arch/x86/include/asm/page_64.h |   16 ++++++++++++++--
>  arch/x86/lib/copy_page_64.S    |   17 +++++++----------
>  2 files changed, 21 insertions(+), 12 deletions(-)
> 
> --- a/arch/x86/include/asm/page_64.h
> +++ b/arch/x86/include/asm/page_64.h
> @@ -49,8 +49,20 @@ static inline void clear_page(void *page)
>  			   : "memory", "rax", "rcx");
>  }
>  
> -void copy_page(void *to, void *from);
> -
> +void copy_page_mov(void *to, void *from);
> +void copy_page_rep_movsq(void *to, void *from);
> +void copy_page_rep_movsb(void *to, void *from);

<---- newline here.

> +static __always_inline void copy_page(void *to, void *from)
> +{
> +	alternative_call_2(

Please align at the opening brace, like clear_page() above it:

	alternative_call_2(copy_page_mov,
			   copy_page_rep_movsq, X86_FEATURE_REP_GOOD,
			   ...


> +		copy_page_rep_movsb, X86_FEATURE_ERMS,
> +		ASM_OUTPUT2("=D" (to), "=S" (from)),
> +		"0" (to), "1" (from)
> +		: "rax", "rcx", "rdx", "r8", "r9", "r10", "r11", "cc", "memory"
> +	);
> +}
>  #endif	/* !__ASSEMBLY__ */
>  
>  #ifdef CONFIG_X86_VSYSCALL_EMULATION

...

>  ENTRY(copy_page_rep_movsb)
>  	mov	$4096, %ecx
>  	rep movsb
>  	ret
>  ENDPROC(copy_page_rep_movsb)
> +EXPORT_SYMBOL(copy_page_rep_movsb)
>  
> -ENTRY(copy_page_regs)
> +ENTRY(copy_page_mov)

copy_page_regs() is a better name IMO. copy_page_mov() doesn't tell me
anything - all three use "mov". copy_page_unrolled() sounds ok too.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.