Date:   Wed, 3 Feb 2021 14:17:11 +0000
From:   "Zhang, Rui" <rui.zhang@...el.com>
To:     Peter Zijlstra <peterz@...radead.org>
CC:     "mingo@...hat.com" <mingo@...hat.com>,
        "acme@...nel.org" <acme@...nel.org>,
        "mark.rutland@....com" <mark.rutland@....com>,
        "alexander.shishkin@...ux.intel.com" 
        <alexander.shishkin@...ux.intel.com>,
        "jolsa@...hat.com" <jolsa@...hat.com>,
        "namhyung@...nel.org" <namhyung@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "x86@...nel.org" <x86@...nel.org>,
        "kan.liang@...ux.intel.com" <kan.liang@...ux.intel.com>,
        "ak@...ux.intel.com" <ak@...ux.intel.com>
Subject: RE: [PATCH 3/3] perf/x86/rapl: Fix psys-energy event on Intel SPR
 platform

Hi, Peter,

> -----Original Message-----
> From: Zhang, Rui
> Sent: Monday, January 25, 2021 2:11 PM
> To: 'Peter Zijlstra' <peterz@...radead.org>
> Cc: 'mingo@...hat.com' <mingo@...hat.com>; 'acme@...nel.org'
> <acme@...nel.org>; 'mark.rutland@....com' <mark.rutland@....com>;
> 'alexander.shishkin@...ux.intel.com' <alexander.shishkin@...ux.intel.com>;
> 'jolsa@...hat.com' <jolsa@...hat.com>; 'namhyung@...nel.org'
> <namhyung@...nel.org>; 'linux-kernel@...r.kernel.org' <linux-
> kernel@...r.kernel.org>; 'x86@...nel.org' <x86@...nel.org>;
> 'kan.liang@...ux.intel.com' <kan.liang@...ux.intel.com>; 'ak@...ux.intel.com'
> <ak@...ux.intel.com>
> Subject: RE: [PATCH 3/3] perf/x86/rapl: Fix psys-energy event on Intel SPR
> platform
> 
> Hi, Peter,
> 
> > -----Original Message-----
> > From: Zhang, Rui
> > Sent: Sunday, January 17, 2021 10:34 PM
> > To: 'Peter Zijlstra' <peterz@...radead.org>
> > Cc: mingo@...hat.com; acme@...nel.org; mark.rutland@....com;
> > alexander.shishkin@...ux.intel.com; jolsa@...hat.com;
> > namhyung@...nel.org; linux-kernel@...r.kernel.org; x86@...nel.org;
> > kan.liang@...ux.intel.com; ak@...ux.intel.com
> > Subject: RE: [PATCH 3/3] perf/x86/rapl: Fix psys-energy event on Intel
> > SPR platform
> >
> > Hi, Peter,
> >
> > > -----Original Message-----
> > > From: Peter Zijlstra <peterz@...radead.org>
> > > Sent: Saturday, January 16, 2021 8:50 PM
> > > To: Zhang, Rui <rui.zhang@...el.com>
> > > Cc: mingo@...hat.com; acme@...nel.org; mark.rutland@....com;
> > > alexander.shishkin@...ux.intel.com; jolsa@...hat.com;
> > > namhyung@...nel.org; linux-kernel@...r.kernel.org; x86@...nel.org;
> > > kan.liang@...ux.intel.com; ak@...ux.intel.com
> > > Subject: Re: [PATCH 3/3] perf/x86/rapl: Fix psys-energy event on
> > > Intel SPR platform
> > > Importance: High
> > >
> > > On Fri, Jan 15, 2021 at 05:22:08PM +0800, Zhang Rui wrote:
> > > > There are several things special for the RAPL Psys energy counter,
> > > > on Intel Sapphire Rapids platform.
> > > > 1. it contains one Psys master package, and only CPUs on the master
> > > >    package can read valid value of the Psys energy counter, reading the
> > > >    MSR on CPUs in the slave package returns 0.
> > > > 2. The master package does not have to be Physical package 0. And
> when
> > > >    all the CPUs on the Psys master package are offlined, we lose the Psys
> > > >    energy counter, at runtime.
> > > > 3. The Psys energy counter can be disabled by BIOS, while all the other
> > > >    energy counters are not affected.
> > > >
> > > > It is not easy to handle all of these in the current RAPL PMU
> > > > design because
> > > > a) perf_msr_probe() validates the MSR on some random CPU, which
> > > > may
> > > either
> > > >    be in the Psys master package or in the Psys slave package.
> > > > b) all the RAPL events share the same PMU, and there is not API to
> > remove
> > > >    the psys-energy event cleanly, without affecting the other events in
> > > >    the same PMU.
> > > >
> > > > This patch addresses the problems in a simple way.
> > > >
> > > > First, by setting .no_check bit for RAPL Psys MSR, the psys-energy
> > > > event is always added, so we don't have to check the Psys
> > > > ENERGY_STATUS MSR on master package.
> > > >
> > > > Then, rapl_not_visible() is removed because 1. it is useless for
> > > > RAPL MSRs with .no_check cleared, because the
> > > >    .is_visible() callbacks is always overridden in perf_msr_probe().
> > > > 2. it is useless for RAPL MSRs with .no_check set, because we actually
> > > >    want the sysfs attributes always be visible for those MSRs.
> > > >
> > > > With the above changes, we always probe the psys-energy event on
> > > > Intel SPR platform. Difference is that the event counter returns 0
> > > > when the Psys RAPL Domain is disabled by BIOS, or the Psys master
> > > > package is
> > > offlined.
> > >
> > > Maybe I'm too tired, but I cannot follow. How does this cure the
> > > fact that the rapl_cpu_mask might not include that master thing. And
> > > how can software detect what the master thing is to begin with?
> >
> > To make things simple, I ignore the master thing, and probe the
> > psys-energy counter blindly on SPR.
> > So rapl_cpu_mask still includes all the online CPUs.
> > This means that psys-energy is "valid" on all packages, and it just
> > returns different values on different packages.
> > AKA, whole system power consumption on Psys master package, and Zero
> > on Psys slave packages.
> >
> In short, the current code does not allow RAPL energy counter to return 0.
> And all the work I do is to allow Psys energy counter to return 0.
> In this way, the Psys event is "valid" on all CPUs, so we don't need to handle
> the master thing.
> The drawback is that we still see psys-energy event, but with 0 readout,
> when Psys counter is not available (master package offlined, or psys
> disabled).
> 
> TBH, I'm not quite sure if I understand your original question correctly or not,
> so please let me know if there is still something unclear.
> 
Sorry to bother, may I know your concern about this patch series?

Thanks,
rui
> Thanks,
> rui
> >
> > Thanks,
> > rui

Powered by blists - more mailing lists