| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <14aca9af-e649-19a7-1be5-2389d6f8348b@intel.com>
Date: Thu, 15 Dec 2022 10:30:33 -0800
From: Reinette Chatre <reinette.chatre@...el.com>
To: Babu Moger <babu.moger@....com>, <corbet@....net>,
<tglx@...utronix.de>, <mingo@...hat.com>, <bp@...en8.de>
CC: <fenghua.yu@...el.com>, <dave.hansen@...ux.intel.com>,
<x86@...nel.org>, <hpa@...or.com>, <paulmck@...nel.org>,
<akpm@...ux-foundation.org>, <quic_neeraju@...cinc.com>,
<rdunlap@...radead.org>, <damien.lemoal@...nsource.wdc.com>,
<songmuchun@...edance.com>, <peterz@...radead.org>,
<jpoimboe@...nel.org>, <pbonzini@...hat.com>,
<chang.seok.bae@...el.com>, <pawan.kumar.gupta@...ux.intel.com>,
<jmattson@...gle.com>, <daniel.sneddon@...ux.intel.com>,
<sandipan.das@....com>, <tony.luck@...el.com>,
<james.morse@....com>, <linux-doc@...r.kernel.org>,
<linux-kernel@...r.kernel.org>, <bagasdotme@...il.com>,
<eranian@...gle.com>, <christophe.leroy@...roup.eu>,
<jarkko@...nel.org>, <adrian.hunter@...el.com>,
<quic_jiles@...cinc.com>, <peternewman@...gle.com>
Subject: Re: [PATCH v9 13/13] Documentation/x86: Update resctrl.rst for new
features
Hi Babu,
On 12/1/2022 7:37 AM, Babu Moger wrote:
> Update the documentation for the new features:
> 1. Slow Memory Bandwidth allocation (SMBA).
> With this feature, the QOS enforcement policies can be applied
> to the external slow memory connected to the host. QOS enforcement
> is accomplished by assigning a Class Of Service (COS) to a processor
> and specifying allocations or limits for that COS for each resource
> to be allocated.
>
> 2. Bandwidth Monitoring Event Configuration (BMEC).
> The bandwidth monitoring events mbm_total_bytes and mbm_local_bytes
> are set to count all the total and local reads/writes respectively.
> With the introduction of slow memory, the two counters are not
> enough to count all the different types of memory events. With the
> feature BMEC, the users have the option to configure mbm_total_bytes
> and mbm_local_bytes to count the specific type of events.
>
> Also add configuration instructions with examples.
>
> Signed-off-by: Babu Moger <babu.moger@....com>
> Reviewed-by: Bagas Sanjaya <bagasdotme@...il.com>
> ---
> Documentation/x86/resctrl.rst | 138 ++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 136 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/x86/resctrl.rst b/Documentation/x86/resctrl.rst
> index 71a531061e4e..60761a6f9087 100644
> --- a/Documentation/x86/resctrl.rst
> +++ b/Documentation/x86/resctrl.rst
> @@ -17,14 +17,16 @@ AMD refers to this feature as AMD Platform Quality of Service(AMD QoS).
> This feature is enabled by the CONFIG_X86_CPU_RESCTRL and the x86 /proc/cpuinfo
> flag bits:
>
> -============================================= ================================
> +=============================================== ================================
> RDT (Resource Director Technology) Allocation "rdt_a"
> CAT (Cache Allocation Technology) "cat_l3", "cat_l2"
> CDP (Code and Data Prioritization) "cdp_l3", "cdp_l2"
> CQM (Cache QoS Monitoring) "cqm_llc", "cqm_occup_llc"
> MBM (Memory Bandwidth Monitoring) "cqm_mbm_total", "cqm_mbm_local"
> MBA (Memory Bandwidth Allocation) "mba"
> -============================================= ================================
> +SMBA (Slow Memory Bandwidth Allocation) "smba"
> +BMEC (Bandwidth Monitoring Event Configuration) "bmec"
> +=============================================== ================================
>
> To use the feature mount the file system::
>
> @@ -161,6 +163,79 @@ with the following files:
> "mon_features":
> Lists the monitoring events if
> monitoring is enabled for the resource.
> + Example::
> +
> + # cat /sys/fs/resctrl/info/L3_MON/mon_features
> + llc_occupancy
> + mbm_total_bytes
> + mbm_local_bytes
> +
> + If the system supports Bandwidth Monitoring Event
> + Configuration (BMEC), then the bandwidth events will
> + be configurable. The output will be::
> +
> + # cat /sys/fs/resctrl/info/L3_MON/mon_features
> + llc_occupancy
> + mbm_total_bytes
> + mbm_total_bytes_config
> + mbm_local_bytes
> + mbm_local_bytes_config
> +
> +"mbm_total_bytes_config", "mbm_local_bytes_config":
> + These files contain the current event configuration for the events
"These files" is redundant. Note that this is already introduced with "the
following files:".
To match similar files it could read:
"Read/write files containing the configuration for the mbm_total_bytes and
mbm_local_bytes events, respectively, ..."
> + mbm_total_bytes and mbm_local_bytes, respectively, when the
> + Bandwidth Monitoring Event Configuration (BMEC) feature is supported.
> + The event configuration settings are domain specific and will affect
"will" can be dropped?
> + all the CPUs in the domain.
> +
> + Following are the types of events supported:
> +
> + ==== ========================================================
> + Bits Description
> + ==== ========================================================
> + 6 Dirty Victims from the QOS domain to all types of memory
> + 5 Reads to slow memory in the non-local NUMA domain
> + 4 Reads to slow memory in the local NUMA domain
> + 3 Non-temporal writes to non-local NUMA domain
> + 2 Non-temporal writes to local NUMA domain
> + 1 Reads to memory in the non-local NUMA domain
> + 0 Reads to memory in the local NUMA domain
> + ==== ========================================================
> +
> + By default, the mbm_total_bytes configuration is set to 0x7f to count
> + all the event types and the mbm_local_bytes configuration is set to
> + 0x15 to count all the local memory events.
> +
> + Examples:
> +
> + * To view the current configuration::
> + ::
> +
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config
> + 0=0x7f;1=0x7f;2=0x7f;3=0x7f
> +
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
> + 0=0x15;1=0x15;3=0x15;4=0x15
> +
> + * To change the mbm_total_bytes to count only reads on domain 0,
> + the bits 0, 1, 4 and 5 needs to be set, which is 110011b in binary
> + (in hexadecimal 0x33):
> + ::
> +
> + # echo "0=0x33" > /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config
> +
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config
> + 0=0x33;1=0x7f;2=0x7f;3=0x7f
> +
> + * To change the mbm_local_bytes to count all the slow memory reads on
> + domain 0 and 1, the bits 4 and 5 needs to be set, which is 110000b
> + in binary (in hexadecimal 0x30):
> + ::
> +
> + # echo "0=0x30;1=0x30" > /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
> +
> + # cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
> + 0=0x30;1=0x30;3=0x15;4=0x15
>
> "max_threshold_occupancy":
> Read/write file provides the largest value (in
> @@ -464,6 +539,25 @@ Memory bandwidth domain is L3 cache.
>
> MB:<cache_id0>=bw_MBps0;<cache_id1>=bw_MBps1;...
>
> +Slow Memory Bandwidth Allocation (SMBA)
> +---------------------------------------
> +AMD hardware supports Slow Memory Bandwidth Allocation (SMBA).
> +CXL.memory is the only supported "slow" memory device. With the
> +support of SMBA, the hardware enables bandwidth allocation on
> +the slow memory devices. If there are multiple such devices in
> +the system, the throttling logic groups all the slow sources
> +together and applies the limit on them as a whole.
> +
> +The presence of SMBA (with CXL.memory) is independent of slow memory
> +devices presence. If there are no such devices on the system, then
> +configuring SMBA will have no impact on the performance of the system.
> +
> +The bandwidth domain for slow memory is L3 cache. Its schemata file
> +is formatted as:
> +::
> +
> + SMBA:<cache_id0>=bandwidth0;<cache_id1>=bandwidth1;...
> +
> Reading/writing the schemata file
> ---------------------------------
> Reading the schemata file will show the state of all resources
> @@ -479,6 +573,46 @@ which you wish to change. E.g.
> L3DATA:0=fffff;1=fffff;2=3c0;3=fffff
> L3CODE:0=fffff;1=fffff;2=fffff;3=fffff
>
> +Reading/writing the schemata file (on AMD systems)
> +--------------------------------------------------
> +Reading the schemata file will show the current bandwidth limit on all
> +domains. The allocated resources are in multiples of one eighth GB/s.
> +When writing to the file, you need to specify what cache id you wish to
> +configure the bandwidth limit.
> +
> +For example, to allocate 2GB/s limit on the first cache id:
> +
> +::
> +
> + # cat schemata
> + MB:0=2048;1=2048;2=2048;3=2048
> + L3:0=ffff;1=ffff;2=ffff;3=ffff
> +
> + # echo "MB:1=16" > schemata
> + # cat schemata
> + MB:0=2048;1= 16;2=2048;3=2048
> + L3:0=ffff;1=ffff;2=ffff;3=ffff
> +
> +Reading/writing the schemata file (on AMD systems) with SMBA feature
> +--------------------------------------------------------------------
> +Reading and writing the schemata file is the same as without SMBA in
> +above section.
> +
> +For example, to allocate 8GB/s limit on the first cache id:
> +
> +::
> +
> + # cat schemata
> + SMBA:0=2048;1=2048;2=2048;3=2048
> + MB:0=2048;1=2048;2=2048;3=2048
> + L3:0=ffff;1=ffff;2=ffff;3=ffff
> +
> + # echo "SMBA:1=64" > schemata
> + # cat schemata
> + SMBA:0=2048;1= 64;2=2048;3=2048
> + MB:0=2048;1=2048;2=2048;3=2048
> + L3:0=ffff;1=ffff;2=ffff;3=ffff
> +
> Cache Pseudo-Locking
> ====================
> CAT enables a user to specify the amount of cache space that an
>
>
Based on earlier comments I am awaiting information to understand if some
more detail/example is needed to describe to the user what can be expected
after a counter configuration is made.
Reinette
Powered by blists - more mailing lists