linux-kernel - [tip: timers/urgent] lib/vdso: Force inlining of __cvdso_clock_gettime

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 03 Jun 2020 18:53:45 -0000
From:   "tip-bot2 for Christophe Leroy" <tip-bot2@...utronix.de>
To:     linux-tip-commits@...r.kernel.org
Cc:     Christophe Leroy <christophe.leroy@....fr>,
        Thomas Gleixner <tglx@...utronix.de>, x86 <x86@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: [tip: timers/urgent] lib/vdso: Force inlining of
 __cvdso_clock_gettime_common()

The following commit has been merged into the timers/urgent branch of tip:

Commit-ID:     b91c8c42ffdd5c983923edb38b3c3e112bfe6263
Gitweb:        https://git.kernel.org/tip/b91c8c42ffdd5c983923edb38b3c3e112bfe6263
Author:        Christophe Leroy <christophe.leroy@....fr>
AuthorDate:    Tue, 28 Apr 2020 13:16:53 
Committer:     Thomas Gleixner <tglx@...utronix.de>
CommitterDate: Wed, 03 Jun 2020 20:50:57 +02:00

lib/vdso: Force inlining of __cvdso_clock_gettime_common()

When adding gettime64() to a 32 bit architecture (namely powerpc/32)
it has been noticed that GCC doesn't inline anymore
__cvdso_clock_gettime_common() because it is called twice
(Once by __cvdso_clock_gettime() and once by
__cvdso_clock_gettime32).

This has the effect of seriously degrading the performance:

Before the implementation of gettime64(), gettime() runs in:

  clock-gettime-monotonic-raw:	    vdso: 1003 nsec/call
  clock-gettime-monotonic-coarse:   vdso:  592 nsec/call
  clock-gettime-monotonic:          vdso:  942 nsec/call

When adding a gettime64() entry point, the standard gettime()
performance is degraded by 30% to 50%:

  clock-gettime-monotonic-raw:      vdso: 1300 nsec/call
  clock-gettime-monotonic-coarse:   vdso:  900 nsec/call
  clock-gettime-monotonic:          vdso: 1232 nsec/call

Adding __always_inline() to __cvdso_clock_gettime_common() regains the
original performance.

In terms of code size, the inlining increases the code size by only 176
bytes. This is in the noise for a kernel image.

Signed-off-by: Christophe Leroy <christophe.leroy@....fr>
Signed-off-by: Thomas Gleixner <tglx@...utronix.de>
Link: https://lkml.kernel.org/r/1ab6a62c356c3bec35d1623563ef9c636205bcda.1588079622.git.christophe.leroy@c-s.fr

---
 lib/vdso/gettimeofday.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index a2909af..7938d3c 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -210,7 +210,7 @@ static __always_inline int do_coarse(const struct vdso_data *vd, clockid_t clk,
 	return 0;
 }
 
-static __maybe_unused int
+static __always_inline int
 __cvdso_clock_gettime_common(const struct vdso_data *vd, clockid_t clock,
 			     struct __kernel_timespec *ts)
 {