linux-kernel - Re: [RFC PATCH 5/8] entry: Explicitly flush pending rcuog wakeup before last rescheduling points

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <X/w/qBdnnQ5pFQxJ@hirez.programming.kicks-ass.net>
Date:   Mon, 11 Jan 2021 13:08:08 +0100
From:   Peter Zijlstra <peterz@...radead.org>
To:     Frederic Weisbecker <frederic@...nel.org>
Cc:     "Paul E . McKenney" <paulmck@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        "Rafael J . Wysocki" <rafael.j.wysocki@...el.com>,
        Ingo Molnar <mingo@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>, stable@...r.kernel.org
Subject: Re: [RFC PATCH 5/8] entry: Explicitly flush pending rcuog wakeup
 before last rescheduling points

On Sat, Jan 09, 2021 at 03:05:33AM +0100, Frederic Weisbecker wrote:
> Following the idle loop model, cleanly check for pending rcuog wakeup
> before the last rescheduling point on resuming to user mode. This
> way we can avoid to do it from rcu_user_enter() with the last resort
> self-IPI hack that enforces rescheduling.
> 
> Signed-off-by: Frederic Weisbecker <frederic@...nel.org>
> Cc: Peter Zijlstra <peterz@...radead.org>
> Cc: Thomas Gleixner <tglx@...utronix.de>
> Cc: Ingo Molnar<mingo@...nel.org>
> Cc: Paul E. McKenney <paulmck@...nel.org>
> Cc: Rafael J. Wysocki <rafael.j.wysocki@...el.com>
> ---
>  kernel/entry/common.c |  6 ++++++
>  kernel/rcu/tree.c     | 12 +++++++-----
>  2 files changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/entry/common.c b/kernel/entry/common.c
> index 378341642f94..8f3292b5f9b7 100644
> --- a/kernel/entry/common.c
> +++ b/kernel/entry/common.c
> @@ -178,6 +178,9 @@ static unsigned long exit_to_user_mode_loop(struct pt_regs *regs,
>  		/* Architecture specific TIF work */
>  		arch_exit_to_user_mode_work(regs, ti_work);
>  
> +		/* Check if any of the above work has queued a deferred wakeup */
> +		rcu_nocb_flush_deferred_wakeup();
> +
>  		/*
>  		 * Disable interrupts and reevaluate the work flags as they
>  		 * might have changed while interrupts and preemption was
> @@ -197,6 +200,9 @@ static void exit_to_user_mode_prepare(struct pt_regs *regs)
>  
>  	lockdep_assert_irqs_disabled();
>  
> +	/* Flush pending rcuog wakeup before the last need_resched() check */
> +	rcu_nocb_flush_deferred_wakeup();
> +
>  	if (unlikely(ti_work & EXIT_TO_USER_MODE_WORK))
>  		ti_work = exit_to_user_mode_loop(regs, ti_work);
>  
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 2920dfc9f58c..3c4c0d5cea65 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -705,12 +705,14 @@ noinstr void rcu_user_enter(void)
>  
>  	lockdep_assert_irqs_disabled();
>  	/*
> -	 * We may be past the last rescheduling opportunity in the entry code.
> -	 * Trigger a self IPI that will fire and reschedule once we resume to
> -	 * user/guest mode.
> +	 * Other than generic entry implementation, we may be past the last
> +	 * rescheduling opportunity in the entry code. Trigger a self IPI
> +	 * that will fire and reschedule once we resume in user/guest mode.
>  	 */
> -	if (do_nocb_deferred_wakeup(rdp) && need_resched())
> -		irq_work_queue(this_cpu_ptr(&late_wakeup_work));
> +	if (!IS_ENABLED(CONFIG_GENERIC_ENTRY) || (current->flags & PF_VCPU)) {

We have xfer_to_guest_mode_work() for that PF_VCPU case.

> +		if (do_nocb_deferred_wakeup(rdp) && need_resched())
> +			irq_work_queue(this_cpu_ptr(&late_wakeup_work));
> +	}