[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <5286c498-99a4-4068-8f0e-5d99e18519bf@arm.com>
Date: Mon, 19 Aug 2024 17:35:45 +0100
From: Robin Murphy <robin.murphy@....com>
To: Mark Rutland <mark.rutland@....com>
Cc: will@...nel.org, linux-arm-kernel@...ts.infradead.org,
linux-kernel@...r.kernel.org, ilkka@...amperecomputing.com
Subject: Re: [PATCH 5/8] perf/arm-cmn: Make cycle counts less surprising
On 16/08/2024 11:25 am, Mark Rutland wrote:
> On Fri, Aug 09, 2024 at 08:15:44PM +0100, Robin Murphy wrote:
>> By default, CMN has automatic clock-gating with the implication that a
>> DTC's cycle counter may not increment while the domain is sufficiently
>> idle.
>
> Similar is true for the cycles event on the CPU side, so this has some
> precedent.
>
>> Given that we may have up to 4 DTCs to choose from when scheduling
>> a cycles event, this may potentially lead to surprising results if
>> trying to measure metrics based on activity in a different DTC domain
>> from where cycles end up being counted. Make the reasonable assumption
>> that if the user wants to count cycles, they almost certainly want to
>> count all of the cycles, and disable clock gating while a DTC's cycle
>> counter is in use.
>
> As above, the default does match the CPU side behaviour,
It may end up looking equivalent with only a single DTC domain, but the
effect with multiple DTCs is as if any cpu_cycles event always counted
on CPU0 (or more recently, any random CPU) - that's the real issue here.
> and a user
> might be trying to determine how much clock gating occurs over some
> period, so it's not necessarily right to always disable clock gating.
> That might need to be an explicit option on the cycles event.
I have also been wondering whether there would be any use-case for a
"dtc_active_cycles" event, but it would would have to be targeted at a
specific DTC node ID or domain number to be meaningful, so would still
be a new thing either way, but even then it seems it would be incredibly
niche - you couldn't get an actual measure of idleness without a second
DTC to count ungated cycles over the same period, and it could only be
correlated with targeted single-node events, since we don't have support
or a syntax for aggregating only within a specific DTC domain.
Yes, some of that could be achieved, but it would represent significant
further complication to the driver and interface, in aid of something
which to the best of my knowledge nobody's actually asking for. What
users *have* asked me, though, is why they were seeing inconsistent
dtc_cycles counts, hence this patch.
> Do we always have the ability to disable clock gating, or can that be
> locked down by system integration or FW?
Yes, the entire PMU may be locked down by a secure override, but
otherwise if we do have use of the DTC then there is no limitation on
cg_disable itself.
Thanks,
Robin.
>
> Mark.
>
>>
>> Signed-off-by: Robin Murphy <robin.murphy@....com>
>> ---
>> drivers/perf/arm-cmn.c | 16 +++++++++++-----
>> 1 file changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c
>> index 8f7a1a6f8ab7..4d128db2040c 100644
>> --- a/drivers/perf/arm-cmn.c
>> +++ b/drivers/perf/arm-cmn.c
>> @@ -115,6 +115,7 @@
>> /* The DTC node is where the magic happens */
>> #define CMN_DT_DTC_CTL 0x0a00
>> #define CMN_DT_DTC_CTL_DT_EN BIT(0)
>> +#define CMN_DT_DTC_CTL_CG_DISABLE BIT(10)
>>
>> /* DTC counters are paired in 64-bit registers on a 16-byte stride. Yuck */
>> #define _CMN_DT_CNT_REG(n) ((((n) / 2) * 4 + (n) % 2) * 4)
>> @@ -1544,9 +1545,12 @@ static void arm_cmn_event_start(struct perf_event *event, int flags)
>> int i;
>>
>> if (type == CMN_TYPE_DTC) {
>> - i = hw->dtc_idx[0];
>> - writeq_relaxed(CMN_CC_INIT, cmn->dtc[i].base + CMN_DT_PMCCNTR);
>> - cmn->dtc[i].cc_active = true;
>> + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0];
>> +
>> + writel_relaxed(CMN_DT_DTC_CTL_DT_EN | CMN_DT_DTC_CTL_CG_DISABLE,
>> + dtc->base + CMN_DT_DTC_CTL);
>> + writeq_relaxed(CMN_CC_INIT, dtc->base + CMN_DT_PMCCNTR);
>> + dtc->cc_active = true;
>> } else if (type == CMN_TYPE_WP) {
>> u64 val = CMN_EVENT_WP_VAL(event);
>> u64 mask = CMN_EVENT_WP_MASK(event);
>> @@ -1575,8 +1579,10 @@ static void arm_cmn_event_stop(struct perf_event *event, int flags)
>> int i;
>>
>> if (type == CMN_TYPE_DTC) {
>> - i = hw->dtc_idx[0];
>> - cmn->dtc[i].cc_active = false;
>> + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0];
>> +
>> + dtc->cc_active = false;
>> + writel_relaxed(CMN_DT_DTC_CTL_DT_EN, dtc->base + CMN_DT_DTC_CTL);
>> } else if (type == CMN_TYPE_WP) {
>> for_each_hw_dn(hw, dn, i) {
>> void __iomem *base = dn->pmu_base + CMN_DTM_OFFSET(hw->dtm_offset);
>> --
>> 2.39.2.101.g768bb238c484.dirty
>>
Powered by blists - more mailing lists