linux-kernel - Re: [PATCH v4 7/8] cpuidle/poll_state: replace cpu_relax with smp_cond_load

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <b5ace5416d1a24b510555bfa03aff4cc35a52cb3.camel@amazon.com>
Date: Mon, 8 Apr 2024 20:04:49 +0000
From: "Okanovic, Haris" <harisokn@...zon.com>
To: "ankur.a.arora@...cle.com" <ankur.a.arora@...cle.com>
CC: "joao.m.martins@...cle.com" <joao.m.martins@...cle.com>,
	"kvm@...r.kernel.org" <kvm@...r.kernel.org>, "dianders@...omium.org"
	<dianders@...omium.org>, "rick.p.edgecombe@...el.com"
	<rick.p.edgecombe@...el.com>, "rafael@...nel.org" <rafael@...nel.org>,
	"pmladek@...e.com" <pmladek@...e.com>, "wanpengli@...cent.com"
	<wanpengli@...cent.com>, "akpm@...ux-foundation.org"
	<akpm@...ux-foundation.org>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "catalin.marinas@....com"
	<catalin.marinas@....com>, "mingo@...hat.com" <mingo@...hat.com>,
	"pbonzini@...hat.com" <pbonzini@...hat.com>, "tglx@...utronix.de"
	<tglx@...utronix.de>, "daniel.lezcano@...aro.org"
	<daniel.lezcano@...aro.org>, "arnd@...db.de" <arnd@...db.de>,
	"mihai.carabas@...cle.com" <mihai.carabas@...cle.com>, "will@...nel.org"
	<will@...nel.org>, "hpa@...or.com" <hpa@...or.com>, "peterz@...radead.org"
	<peterz@...radead.org>, "mic@...ikod.net" <mic@...ikod.net>,
	"vkuznets@...hat.com" <vkuznets@...hat.com>,
	"linux-arm-kernel@...ts.infradead.org"
	<linux-arm-kernel@...ts.infradead.org>, "Okanovic, Haris"
	<harisokn@...zon.com>, "linux-pm@...r.kernel.org" <linux-pm@...r.kernel.org>,
	"bp@...en8.de" <bp@...en8.de>, "juerg.haefliger@...onical.com"
	<juerg.haefliger@...onical.com>, "npiggin@...il.com" <npiggin@...il.com>,
	"x86@...nel.org" <x86@...nel.org>
Subject: Re: [PATCH v4 7/8] cpuidle/poll_state: replace cpu_relax with
 smp_cond_load_relaxed

On Mon, 2024-04-08 at 11:46 -0700, Ankur Arora wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
> Okanovic, Haris <harisokn@...zon.com> writes:
> 
> > On Fri, 2024-04-05 at 16:14 -0700, Ankur Arora wrote:
> > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> > > 
> > > 
> > > 
> > > Okanovic, Haris <harisokn@...zon.com> writes:
> > > 
> > > > On Thu, 2024-02-15 at 09:41 +0200, Mihai Carabas wrote:
> > > > > cpu_relax on ARM64 does a simple "yield". Thus we replace it with
> > > > > smp_cond_load_relaxed which basically does a "wfe".
> > > > > 
> > > > > Suggested-by: Peter Zijlstra <peterz@...radead.org>
> > > > > Signed-off-by: Mihai Carabas <mihai.carabas@...cle.com>
> > > > > ---
> > > > >  drivers/cpuidle/poll_state.c | 15 ++++++++++-----
> > > > >  1 file changed, 10 insertions(+), 5 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/cpuidle/poll_state.c b/drivers/cpuidle/poll_state.c
> > > > > index 9b6d90a72601..1e45be906e72 100644
> > > > > --- a/drivers/cpuidle/poll_state.c
> > > > > +++ b/drivers/cpuidle/poll_state.c
> > > > > @@ -13,6 +13,7 @@
> > > > >  static int __cpuidle poll_idle(struct cpuidle_device *dev,
> > > > >                             struct cpuidle_driver *drv, int index)
> > > > >  {
> > > > > +    unsigned long ret;
> > > > >      u64 time_start;
> > > > > 
> > > > >      time_start = local_clock_noinstr();
> > > > > @@ -26,12 +27,16 @@ static int __cpuidle poll_idle(struct cpuidle_device *dev,
> > > > > 
> > > > >              limit = cpuidle_poll_time(drv, dev);
> > > > > 
> > > > > -            while (!need_resched()) {
> > > > > -                    cpu_relax();
> > > > > -                    if (loop_count++ < POLL_IDLE_RELAX_COUNT)
> > > > > -                            continue;
> > > > > -
> > > > > +            for (;;) {
> > > > >                      loop_count = 0;
> > > > > +
> > > > > +                    ret = smp_cond_load_relaxed(&current_thread_info()->flags,
> > > > > +                                                VAL & _TIF_NEED_RESCHED ||
> > > > > +                                                loop_count++ >= POLL_IDLE_RELAX_COUNT);
> > > > 
> > > > Is it necessary to repeat this 200 times with a wfe poll?
> > > 
> > > The POLL_IDLE_RELAX_COUNT is there because on x86 each cpu_relax()
> > > iteration is much shorter.
> > > 
> > > With WFE, it makes less sense.
> > > 
> > > > Does kvm not implement a timeout period?
> > > 
> > > Not yet, but it does become more useful after a WFE haltpoll is
> > > available on ARM64.
> > 
> > Note that kvm conditionally traps WFE and WFI based on number of host
> > CPU tasks. VMs will sometimes see hardware behavior - potentially
> > polling for a long time before entering WFI.
> > 
> > https://elixir.bootlin.com/linux/latest/source/arch/arm64/kvm/arm.c#L459
> 
> Yeah. There was a discussion on this
> https://lore.kernel.org/lkml/871qc6qufy.fsf@oracle.com/.
> 
> > > Haltpoll does have a timeout, which you should be able to tune via
> > > /sys/module/haltpoll/parameters/ but that, of course, won't help here.
> > > 
> > > > Could you make it configurable? This patch improves certain workloads
> > > > on AWS Graviton instances as well, but blocks up to 6ms in 200 * 30us
> > > > increments before going to wfi, which is a bit excessive.
> > > 
> > > Yeah, this looks like a problem. We could solve it by making it an
> > > architectural parameter. Though I worry about ARM platforms with
> > > much smaller default timeouts.
> > > The other possibility is using WFET in the primitive, but then we
> > > have that dependency and that's a bigger change.
> > 
> > See arm64's delay() for inspiration:
> > 
> > https://elixir.bootlin.com/linux/v6.9-rc2/source/arch/arm64/lib/delay.c#L26
> 
> Sure, that part is straight-forward enough. However, this will need a fallback
> the case when WFET is not available. And, because this path is used on x86,
> so we need a cross platform smp_cond*timeout(). Though given that the x86
> version is based on cpu_relax() then that could just fold the sched_clock()
> check in.

I was trying to point out how delay() handles different configurations:
It prefers WFET when available, falls back to WFE when event stream is
available, and finally falls back to cpu_relax() as last resort. Same
logic can apply here. The x86 case can always use cpu_relax() fallback,
for same behavior as smp_cond_load_relaxed().

Re your concern about "ARM platforms with much smaller default
timeouts": You could do something different when arch_timer_get_rate()
is too small. Although I'm not sure this is a huge concern, given that
delay() doesn't seem to care in the WFE case.

-- Haris Okanovic

> 
> Maybe another place to do this would be by KVM forcing a WFE timeout. Arguably
> that is needed regardless of whether we use a smp_cond*timeout() or not.
> 
> --
> ankur