Message-ID: <20241211143428.kaoovhiwar74dy6x@lcpd911>
Date: Wed, 11 Dec 2024 20:04:28 +0530
From: Dhruva Gole <d-gole@...com>
To: Sudeep Holla <sudeep.holla@....com>
CC: Vivek yadav <linux.ninja23@...il.com>, <linux-newbie@...r.kernel.org>,
        <linux-pm@...r.kernel.org>, <daniel.lezcano@...aro.org>,
        <lpieralisi@...nel.org>, <krzk@...nel.org>, <christian.loehle@....com>,
        <quic_sibis@...cinc.com>, <cristian.marussi@....com>,
        <linux-arm-kernel@...ts.infradead.org>, <linux-kernel@...r.kernel.org>,
        <vigneshr@...com>, <khilman@...com>, <sebin.francis@...com>,
        <khilman@...libre.com>
Subject: Re: Fwd: ARM64: CPUIdle driver is not select any Idle state other
 then WFI

On Dec 11, 2024 at 12:18:25 +0000, Sudeep Holla wrote:
> On Wed, Dec 11, 2024 at 11:20:52AM +0530, Dhruva Gole wrote:
[...]
> > >
> > >
> > > Hi @all,
> > >
> > > I am working on one custom SoC. Where I add one CPUIdle state for
> > > ``arm,cortex-a55`` processor.
> >
> > Any further luck on this?
> >
> > I have also been working on something similar[1] but on an A53 core on
> > TI-K3 AM62x processor.
> 
> Does upstream DTS have support for this platform to understand it better ?
> Even reference to any complete DT file for the platform will help.

Yes, you can ref to the AM625 (CPU layout) DT here:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/arm64/boot/dts/ti/k3-am625.dtsi

The board/starter kit DT is:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/arm64/boot/dts/ti/k3-am625-sk.dts

The patches for idle state are not upstream, and only exist in this
patch of mine here:
https://github.com/DhruvaG2000/v-linux/commit/0fd088d624276a2e72b8dc6660d261ab6d194f4b

[...]
> > See this chunk in the kernel cpuidle driver:
> > 	if (broadcast && tick_broadcast_enter()) {
> >
> > When I dug deeper into tick_broadcast_enter it always returns something
> > non zero and hence in my case it was entering the if block and tried to
> > find a deepest state. Then the deepest state would always return WFI and
> > not the idle-state I had added.
> >
> > What we found out was on our kernel we end up using
> >
> > kernel/time/tick-broadcast-hrtimer.c
> >
> > This always seems to be keeping atleast 1 CPU busy and prevents idle.
> > If we remove the local-timer-stop it was helping us, but we still need
> > to dig into the full impact of what that entails and I am still
> > interested in finding out how so many other users of similar idle-state
> > implementation are able to do so without trouble.
> >
> 
> Interesting. So if the platform is functional removing local-timer-stop,
> I am bit confused. Either there is something else that is getting it out

Yes it was interesting to us too, as to how the RCU didn't kick in and
system continued to function as though nothing was wrong.

> from the idle state so, it should be fine and it could be just some

It's probably UART keypresses or some userspace processes that get
scheduled that bring the CPUs back out of TF-A's cpu_standby.
Is it possible that EL1 interrupts can bring EL3 out of WFI? Is yes then
it explains the behaviour. The arch timer could also be continuing to
tick and bringing the CPUs out of ATF WFI.

> misconfiguration.
> 
> > Arm64 recommends to use arch_timer instead of external timers. Once we
> > enter el3, timer interrupts to el1 is blocked and hence it's equivalent
> > to local-timer-stop, so it does make sense to keep this property, but
> > then how are others able to enter idle-states for all plugged CPUs at
> > the same time?
> >
> 
> Some systems have system timer that can take over as broadcast timer when
> CPUs enter deeper idle states where the local timers are stopped.

In CPUIdle we're not really clock gating anything so the timer does keep
ticking. So in this particular case it might make sense to remove the
local-timer-stop property from the idle-state.

However we're looking into taking this further and putting interconnect
and few other PLLs in bypass which could cause arch timer for eg. to
tick slower. In this case would it still make sense to omit the
property? We may even have some usecases planned where we may turn OFF
the CPU once it is in TF-A cpu_standby/ WFI. What would be the right
approach in such scenarios?

Could you provide any examples where the local-timer-stop property is
being used and an alternative timer can be configured once we enter the
idle-state where CPU CTX maybe lost or clocks maybe bypass? It would be
great if you could share some example implementation if you're aware.

I took a look at QCom / NXP DT's but I couldn't exactly figure out how
they were setting up alternate timers once CPU hit idle-state.

PS. If it helps atall here's also my hack TF-A code where we enter just
CPU RET state for now:
https://github.com/DhruvaG2000/tfa-dev/commit/2d4b441d2f4c6d9ee0d6a62d93920ee8ab77dd42#diff-4fa4f4f6a5faa221390928a8079f76392536559f66e420a0182dba8e5966f4c6R232

-- 
Best regards,
Dhruva Gole
Texas Instruments Incorporated

Powered by blists - more mailing lists