[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <aada0beae0b3479bfa311eea94a3b595bb8e5835.camel@amazon.com>
Date: Sat, 6 Apr 2024 18:42:38 +0000
From: "Okanovic, Haris" <harisokn@...zon.com>
To: "ankur.a.arora@...cle.com" <ankur.a.arora@...cle.com>
CC: "joao.m.martins@...cle.com" <joao.m.martins@...cle.com>,
"kvm@...r.kernel.org" <kvm@...r.kernel.org>, "dianders@...omium.org"
<dianders@...omium.org>, "linux-arm-kernel@...ts.infradead.org"
<linux-arm-kernel@...ts.infradead.org>, "pmladek@...e.com"
<pmladek@...e.com>, "wanpengli@...cent.com" <wanpengli@...cent.com>,
"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"catalin.marinas@....com" <catalin.marinas@....com>, "mingo@...hat.com"
<mingo@...hat.com>, "pbonzini@...hat.com" <pbonzini@...hat.com>,
"tglx@...utronix.de" <tglx@...utronix.de>, "daniel.lezcano@...aro.org"
<daniel.lezcano@...aro.org>, "mihai.carabas@...cle.com"
<mihai.carabas@...cle.com>, "arnd@...db.de" <arnd@...db.de>,
"will@...nel.org" <will@...nel.org>, "hpa@...or.com" <hpa@...or.com>,
"peterz@...radead.org" <peterz@...radead.org>, "mic@...ikod.net"
<mic@...ikod.net>, "vkuznets@...hat.com" <vkuznets@...hat.com>,
"bp@...en8.de" <bp@...en8.de>, "npiggin@...il.com" <npiggin@...il.com>,
"linux-pm@...r.kernel.org" <linux-pm@...r.kernel.org>, "rafael@...nel.org"
<rafael@...nel.org>, "juerg.haefliger@...onical.com"
<juerg.haefliger@...onical.com>, "x86@...nel.org" <x86@...nel.org>,
"rick.p.edgecombe@...el.com" <rick.p.edgecombe@...el.com>
Subject: Re: [PATCH v4 7/8] cpuidle/poll_state: replace cpu_relax with
smp_cond_load_relaxed
On Fri, 2024-04-05 at 16:14 -0700, Ankur Arora wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> Okanovic, Haris <harisokn@...zon.com> writes:
>
> > On Thu, 2024-02-15 at 09:41 +0200, Mihai Carabas wrote:
> > > cpu_relax on ARM64 does a simple "yield". Thus we replace it with
> > > smp_cond_load_relaxed which basically does a "wfe".
> > >
> > > Suggested-by: Peter Zijlstra <peterz@...radead.org>
> > > Signed-off-by: Mihai Carabas <mihai.carabas@...cle.com>
> > > ---
> > > drivers/cpuidle/poll_state.c | 15 ++++++++++-----
> > > 1 file changed, 10 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/cpuidle/poll_state.c b/drivers/cpuidle/poll_state.c
> > > index 9b6d90a72601..1e45be906e72 100644
> > > --- a/drivers/cpuidle/poll_state.c
> > > +++ b/drivers/cpuidle/poll_state.c
> > > @@ -13,6 +13,7 @@
> > > static int __cpuidle poll_idle(struct cpuidle_device *dev,
> > > struct cpuidle_driver *drv, int index)
> > > {
> > > + unsigned long ret;
> > > u64 time_start;
> > >
> > > time_start = local_clock_noinstr();
> > > @@ -26,12 +27,16 @@ static int __cpuidle poll_idle(struct cpuidle_device *dev,
> > >
> > > limit = cpuidle_poll_time(drv, dev);
> > >
> > > - while (!need_resched()) {
> > > - cpu_relax();
> > > - if (loop_count++ < POLL_IDLE_RELAX_COUNT)
> > > - continue;
> > > -
> > > + for (;;) {
> > > loop_count = 0;
> > > +
> > > + ret = smp_cond_load_relaxed(¤t_thread_info()->flags,
> > > + VAL & _TIF_NEED_RESCHED ||
> > > + loop_count++ >= POLL_IDLE_RELAX_COUNT);
> >
> > Is it necessary to repeat this 200 times with a wfe poll?
>
> The POLL_IDLE_RELAX_COUNT is there because on x86 each cpu_relax()
> iteration is much shorter.
>
> With WFE, it makes less sense.
>
> > Does kvm not implement a timeout period?
>
> Not yet, but it does become more useful after a WFE haltpoll is
> available on ARM64.
Note that kvm conditionally traps WFE and WFI based on number of host
CPU tasks. VMs will sometimes see hardware behavior - potentially
polling for a long time before entering WFI.
https://elixir.bootlin.com/linux/latest/source/arch/arm64/kvm/arm.c#L459
>
> Haltpoll does have a timeout, which you should be able to tune via
> /sys/module/haltpoll/parameters/ but that, of course, won't help here.
>
> > Could you make it configurable? This patch improves certain workloads
> > on AWS Graviton instances as well, but blocks up to 6ms in 200 * 30us
> > increments before going to wfi, which is a bit excessive.
>
> Yeah, this looks like a problem. We could solve it by making it an
> architectural parameter. Though I worry about ARM platforms with
> much smaller default timeouts.
> The other possibility is using WFET in the primitive, but then we
> have that dependency and that's a bigger change.
See arm64's delay() for inspiration:
https://elixir.bootlin.com/linux/v6.9-rc2/source/arch/arm64/lib/delay.c#L26
>
> Will address this in the next version.
>
> Thanks for pointing this out.
>
> --
> ankur
Powered by blists - more mailing lists