[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <MW3PR12MB455398D9564E7C3B99D58AF5955F9@MW3PR12MB4553.namprd12.prod.outlook.com>
Date: Fri, 7 Oct 2022 15:51:57 +0000
From: "Moger, Babu" <Babu.Moger@....com>
To: Bagas Sanjaya <bagasdotme@...il.com>,
"corbet@....net" <corbet@....net>,
"reinette.chatre@...el.com" <reinette.chatre@...el.com>,
"tglx@...utronix.de" <tglx@...utronix.de>,
"mingo@...hat.com" <mingo@...hat.com>,
"bp@...en8.de" <bp@...en8.de>
CC: "fenghua.yu@...el.com" <fenghua.yu@...el.com>,
"dave.hansen@...ux.intel.com" <dave.hansen@...ux.intel.com>,
"x86@...nel.org" <x86@...nel.org>, "hpa@...or.com" <hpa@...or.com>,
"paulmck@...nel.org" <paulmck@...nel.org>,
"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
"quic_neeraju@...cinc.com" <quic_neeraju@...cinc.com>,
"rdunlap@...radead.org" <rdunlap@...radead.org>,
"damien.lemoal@...nsource.wdc.com" <damien.lemoal@...nsource.wdc.com>,
"songmuchun@...edance.com" <songmuchun@...edance.com>,
"peterz@...radead.org" <peterz@...radead.org>,
"jpoimboe@...nel.org" <jpoimboe@...nel.org>,
"pbonzini@...hat.com" <pbonzini@...hat.com>,
"chang.seok.bae@...el.com" <chang.seok.bae@...el.com>,
"pawan.kumar.gupta@...ux.intel.com"
<pawan.kumar.gupta@...ux.intel.com>,
"jmattson@...gle.com" <jmattson@...gle.com>,
"daniel.sneddon@...ux.intel.com" <daniel.sneddon@...ux.intel.com>,
"Das1, Sandipan" <Sandipan.Das@....com>,
"tony.luck@...el.com" <tony.luck@...el.com>,
"james.morse@....com" <james.morse@....com>,
"linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"eranian@...gle.com" <eranian@...gle.com>
Subject: RE: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features
[AMD Official Use Only - General]
Hi Sanjaya,
> -----Original Message-----
> From: Bagas Sanjaya <bagasdotme@...il.com>
> Sent: Friday, October 7, 2022 3:33 AM
> To: Moger, Babu <Babu.Moger@....com>; corbet@....net;
> reinette.chatre@...el.com; tglx@...utronix.de; mingo@...hat.com;
> bp@...en8.de
> Cc: fenghua.yu@...el.com; dave.hansen@...ux.intel.com; x86@...nel.org;
> hpa@...or.com; paulmck@...nel.org; akpm@...ux-foundation.org;
> quic_neeraju@...cinc.com; rdunlap@...radead.org;
> damien.lemoal@...nsource.wdc.com; songmuchun@...edance.com;
> peterz@...radead.org; jpoimboe@...nel.org; pbonzini@...hat.com;
> chang.seok.bae@...el.com; pawan.kumar.gupta@...ux.intel.com;
> jmattson@...gle.com; daniel.sneddon@...ux.intel.com; Das1, Sandipan
> <Sandipan.Das@....com>; tony.luck@...el.com; james.morse@....com;
> linux-doc@...r.kernel.org; linux-kernel@...r.kernel.org;
> eranian@...gle.com
> Subject: Re: [PATCH v5 00/12] x86/resctrl: Support for AMD QoS new features
>
> On 9/28/22 03:25, Babu Moger wrote:
> > New AMD processors can now support following QoS features.
> >
> > 1. Slow Memory Bandwidth Allocation (SMBA)
> > With this feature, the QOS enforcement policies can be applied
> > to the external slow memory connected to the host. QOS enforcement
> > is accomplished by assigning a Class Of Service (COS) to a processor
> > and specifying allocations or limits for that COS for each resource
> > to be allocated.
> >
> > Currently, CXL.memory is the only supported "slow" memory device. With
> > the support of SMBA feature the hardware enables bandwidth allocation
> > on the slow memory devices.
> >
> > 2. Bandwidth Monitoring Event Configuration (BMEC)
> > The bandwidth monitoring events mbm_total_event and mbm_local_event
> > are set to count all the total and local reads/writes respectively.
> > With the introduction of slow memory, the two counters are not enough
> > to count all the different types are memory events. With the feature
> > BMEC, the users have the option to configure mbm_total_event and
> > mbm_local_event to count the specific type of events.
> >
> > Following are the bitmaps of events supported.
> > Bits Description
> > 6 Dirty Victims from the QOS domain to all types of memory
> > 5 Reads to slow memory in the non-local NUMA domain
> > 4 Reads to slow memory in the local NUMA domain
> > 3 Non-temporal writes to non-local NUMA domain
> > 2 Non-temporal writes to local NUMA domain
> > 1 Reads to memory in the non-local NUMA domain
> > 0 Reads to memory in the local NUMA domain
> >
> > This series adds support for these features.
> >
> > Feature description is available in the specification, "AMD64
> > Technology Platform Quality of Service Extensions, Revision: 1.03 Publication
> # 56375 Revision: 1.03 Issue Date: February 2022".
> >
> > Link:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> > amd.com%2Fen%2Fsupport%2Ftech-docs%2Famd64-technology-platform-
> quality
> > -service-
> extensions&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc806
> >
> 9ca2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%
> 7C0%7C
> >
> 638007284215759374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> AiLCJQIjo
> >
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdat
> a=qAZS
> > ze2Mbg24Z0%2BX0GN4yrVO2ooQqQyum7NUjwIGg5o%3D&reserved=0
> > Link:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugz
> >
> illa.kernel.org%2Fshow_bug.cgi%3Fid%3D206537&data=05%7C01%7Cbab
> u.m
> >
> oger%40amd.com%7Cda5fc8069ca2484b2aef08daa83ea08a%7C3dd8961fe488
> 4e608e
> >
> 11a82d994e183d%7C0%7C0%7C638007284215759374%7CUnknown%7CTWFpb
> GZsb3d8ey
> >
> JWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> C300
> >
> 0%7C%7C%7C&sdata=8%2BFbGTVfkp%2BCbyyk%2BYa9u0JiHi2YZEVaHiUs
> CBw335g
> > %3D&reserved=0
> >
> > ---
> > v5:
> > Summary of changes.
> > 1. Split the series into two. The first two patches are bug fixes. So, sent them
> separate.
> > 2. The config files mbm_total_config and mbm_local_config are now under
> > /sys/fs/resctrl/info/L3_MON/. Removed these config files from mon
> groups.
> > 3. Ran "checkpatch --strict --codespell" on all the patches. Looks good with
> few known exceptions.
> > 4. Few minor text changes in resctrl.rst file.
> >
> > v4:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> el.org%2Flkml%2F166257348081.1043018.11227924488792315932.stgit%40bm
> oger-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638007284215759374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=qamR1M2sVSo4vE9NVZzFmvIop7YMKNIYHT74NJrbVVk%3D&am
> p;reserved=0
> > Got numerios of comments from Reinette Chatre. Addressed most of them.
> > Summary of changes.
> > 1. Removed mon_configurable under /sys/fs/resctrl/info/L3_MON/.
> > 2. Updated mon_features texts if the BMEC is supported.
> > 3. Added more explanation about the slow memory support.
> > 4. Replaced smp_call_function_many with on_each_cpu_mask call.
> > 5. Removed arch_has_empty_bitmaps
> > 6. Few other text changes.
> > 7. Removed Reviewed-by if the patch is modified.
> > 8. Rebased the patches to latest tip.
> >
> > v3:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> el.org%2Flkml%2F166117559756.6695.16047463526634290701.stgit%40bmoge
> r-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=ogINZslT9yExkkFww4X14XQEg8W8heYBrJB59C50Hqk%3D&
> reserved=0
> > a. Rebased the patches to latest tip. Resolved some conflicts.
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kerne
> l.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftip%2Ftip.git&data=05%
> 7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca2484b2aef08daa83ea08a%
> 7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638007284215915604
> %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
> BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZOK3iwsaq
> 3%2BGAUIXJwn9Thg3cJBxWDMAfu4UqHlo2J4%3D&reserved=0
> > b. Taken care of feedback from Bagas Sanjaya.
> > c. Added Reviewed by from Mingo.
> > Note: I am still looking for comments from Reinette or Fenghua.
> >
> > v2:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> el.org%2Flkml%2F165938717220.724959.10931629283087443782.stgit%40bmo
> ger-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=14viyG9elsn6BYGpDOwrqYQFNOlpyC6oqoJwJm49oV0%3D&
> reserved=0
> > a. Rebased the patches to latest stable tree (v5.18.15). Resolved some
> conflicts.
> > b. Added the patch to fix CBM issue on AMD. This was originally discussed
> >
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> > .kernel.org%2Flkml%2F20220517001234.3137157-1-
> eranian%40google.com%2F&
> >
> amp;data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca2484b2aef0
> 8daa83e
> >
> a08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C6380072842159
> 15604%7
> >
> CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> 6Ik1
> >
> haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BHEvN2PNNYyH
> ohvLg2sbth
> > BQo4cgDj5Vsw9AqGb1Pr8%3D&reserved=0
> >
> > v1:
> >
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> >
> .kernel.org%2Flkml%2F165757543252.416408.13547339307237713464.stgit%4
> 0
> > bmoger-
> ubuntu%2F&data=05%7C01%7Cbabu.moger%40amd.com%7Cda5fc8069ca
> >
> 2484b2aef08daa83ea08a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0
> %7C638
> >
> 007284215915604%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> JQIjoiV2
> >
> luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J
> R03cAA
> > 9pzmq0SgNsgAsajOH6LX%2F4s3px3b%2FJ8409Ys%3D&reserved=0
> >
> > Babu Moger (12):
> > x86/cpufeatures: Add Slow Memory Bandwidth Allocation feature flag
> > x86/resctrl: Add a new resource type RDT_RESOURCE_SMBA
> > x86/cpufeatures: Add Bandwidth Monitoring Event Configuration feature
> flag
> > x86/resctrl: Include new features in command line options
> > x86/resctrl: Detect and configure Slow Memory Bandwidth allocation
> > x86/resctrl: Introduce data structure to support monitor configuration
> > x86/resctrl: Add sysfs interface to read mbm_total_bytes event
> configuration
> > x86/resctrl: Add sysfs interface to read mbm_local_bytes event
> configuration
> > x86/resctrl: Add sysfs interface to write mbm_total_bytes event
> configuration
> > x86/resctrl: Add sysfs interface to write mbm_local_bytes event
> configuration
> > x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask()
> > Documentation/x86: Update resctrl_ui.rst for new features
> >
> >
> > .../admin-guide/kernel-parameters.txt | 2 +-
> > Documentation/x86/resctrl.rst | 130 +++++++-
> > arch/x86/include/asm/cpufeatures.h | 2 +
> > arch/x86/kernel/cpu/cpuid-deps.c | 1 +
> > arch/x86/kernel/cpu/resctrl/core.c | 51 ++-
> > arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +-
> > arch/x86/kernel/cpu/resctrl/internal.h | 33 +-
> > arch/x86/kernel/cpu/resctrl/monitor.c | 9 +-
> > arch/x86/kernel/cpu/resctrl/rdtgroup.c | 298 ++++++++++++++++--
> > arch/x86/kernel/cpu/scattered.c | 2 +
> > 10 files changed, 496 insertions(+), 34 deletions(-)
> >
>
> Hi Babu, sorry for having to do public reply to this v5 cover letter due to
> accidentally delete the preview documentation patch for your upcoming v6.
>
> Thanks for privately sending me the preview patch. Seeing it at a glance,
> LGTM. Please send the full v6 series for us to review.
Sure. Will send the whole series.
Thanks
Babu
Powered by blists - more mailing lists