[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ZhX71JRK0W+BaeXR@tpad>
Date: Tue, 9 Apr 2024 23:39:16 -0300
From: Marcelo Tosatti <mtosatti@...hat.com>
To: Sean Christopherson <seanjc@...gle.com>
Cc: "Paul E. McKenney" <paulmck@...nel.org>,
Leonardo Bras <leobras@...hat.com>,
Paolo Bonzini <pbonzini@...hat.com>,
Frederic Weisbecker <frederic@...nel.org>,
Neeraj Upadhyay <quic_neeraju@...cinc.com>,
Joel Fernandes <joel@...lfernandes.org>,
Josh Triplett <josh@...htriplett.org>,
Boqun Feng <boqun.feng@...il.com>,
Steven Rostedt <rostedt@...dmis.org>,
Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
Lai Jiangshan <jiangshanlai@...il.com>,
Zqiang <qiang.zhang1211@...il.com>, kvm@...r.kernel.org,
linux-kernel@...r.kernel.org, rcu@...r.kernel.org
Subject: Re: [RFC PATCH v1 0/2] Avoid rcu_core() if CPU just left guest vcpu
On Mon, Apr 08, 2024 at 10:16:24AM -0700, Sean Christopherson wrote:
> On Fri, Apr 05, 2024, Paul E. McKenney wrote:
> > On Fri, Apr 05, 2024 at 07:42:35AM -0700, Sean Christopherson wrote:
> > > On Fri, Apr 05, 2024, Marcelo Tosatti wrote:
> > > > rcuc wakes up (which might exceed the allowed latency threshold
> > > > for certain realtime apps).
> > >
> > > Isn't that a false negative? (RCU doesn't detect that a CPU is about to (re)enter
> > > a guest) I was trying to ask about the case where RCU thinks a CPU is about to
> > > enter a guest, but the CPU never does (at least, not in the immediate future).
> > >
> > > Or am I just not understanding how RCU's kthreads work?
> >
> > It is quite possible that the current rcu_pending() code needs help,
> > given the possibility of vCPU preemption. I have heard of people doing
> > nested KVM virtualization -- or is that no longer a thing?
>
> Nested virtualization is still very much a thing, but I don't see how it is at
> all unique with respect to RCU grace periods and quiescent states. More below.
>
> > But the help might well involve RCU telling the hypervisor that a given
> > vCPU needs to run. Not sure how that would go over, though it has been
> > prototyped a couple times in the context of RCU priority boosting.
> >
> > > > > > 3 - It checks if the guest exit happened over than 1 second ago. This 1
> > > > > > second value was copied from rcu_nohz_full_cpu() which checks if the
> > > > > > grace period started over than a second ago. If this value is bad,
> > > > > > I have no issue changing it.
> > > > >
> > > > > IMO, checking if a CPU "recently" ran a KVM vCPU is a suboptimal heuristic regardless
> > > > > of what magic time threshold is used.
> > > >
> > > > Why? It works for this particular purpose.
> > >
> > > Because maintaining magic numbers is no fun, AFAICT the heurisitic doesn't guard
> > > against edge cases, and I'm pretty sure we can do better with about the same amount
> > > of effort/churn.
> >
> > Beyond a certain point, we have no choice. How long should RCU let
> > a CPU run with preemption disabled before complaining? We choose 21
> > seconds in mainline and some distros choose 60 seconds. Android chooses
> > 20 milliseconds for synchronize_rcu_expedited() grace periods.
>
> Issuing a warning based on an arbitrary time limit is wildly different than using
> an arbitrary time window to make functional decisions. My objection to the "assume
> the CPU will enter a quiescent state if it exited a KVM guest in the last second"
> is that there are plenty of scenarios where that assumption falls apart, i.e. where
> _that_ physical CPU will not re-enter the guest.
>
> Off the top of my head:
>
> - If the vCPU is migrated to a different physical CPU (pCPU), the *old* pCPU
> will get false positives, and the *new* pCPU will get false negatives (though
> the false negatives aren't all that problematic since the pCPU will enter a
> quiescent state on the next VM-Enter.
>
> - If the vCPU halts, in which case KVM will schedule out the vCPU/task, i.e.
> won't re-enter the guest. And so the pCPU will get false positives until the
> vCPU gets a wake event or the 1 second window expires.
>
> - If the VM terminates, the pCPU will get false positives until the 1 second
> window expires.
>
> The false positives are solvable problems, by hooking vcpu_put() to reset
> kvm_last_guest_exit. And to help with the false negatives when a vCPU task is
> scheduled in on a different pCPU, KVM would hook vcpu_load().
Sean,
It seems that fixing the problems you pointed out above is a way to go.
Powered by blists - more mailing lists