[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <97a45686-3b54-489f-b0f4-847b99312aa0@linux.intel.com>
Date: Thu, 22 May 2025 16:50:58 -0700
From: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@...ux.intel.com>
To: Bjorn Helgaas <helgaas@...nel.org>, linux-pci@...r.kernel.org
Cc: Jon Pan-Doh <pandoh@...gle.com>,
Karolina Stolarek <karolina.stolarek@...cle.com>,
Weinan Liu <wnliu@...gle.com>, Martin Petersen <martin.petersen@...cle.com>,
Ben Fuller <ben.fuller@...cle.com>, Drew Walton <drewwalton@...rosoft.com>,
Anil Agrawal <anilagrawal@...a.com>, Tony Luck <tony.luck@...el.com>,
Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>,
Lukas Wunner <lukas@...ner.de>,
Jonathan Cameron <Jonathan.Cameron@...wei.com>,
Sargun Dhillon <sargun@...a.com>, "Paul E . McKenney" <paulmck@...nel.org>,
Mahesh J Salgaonkar <mahesh@...ux.ibm.com>,
Oliver O'Halloran <oohall@...il.com>, Kai-Heng Feng <kaihengf@...dia.com>,
Keith Busch <kbusch@...nel.org>, Robert Richter <rrichter@....com>,
Terry Bowman <terry.bowman@....com>, Shiju Jose <shiju.jose@...wei.com>,
Dave Jiang <dave.jiang@...el.com>, linux-kernel@...r.kernel.org,
linuxppc-dev@...ts.ozlabs.org, Bjorn Helgaas <bhelgaas@...gle.com>
Subject: Re: [PATCH v8 20/20] PCI/AER: Add sysfs attributes for log ratelimits
Hi,
On 5/22/25 4:21 PM, Bjorn Helgaas wrote:
> From: Jon Pan-Doh <pandoh@...gle.com>
>
> Allow userspace to read/write log ratelimits per device (including
> enable/disable). Create aer/ sysfs directory to store them and any
> future AER configs.
>
> The new sysfs files are:
>
> /sys/bus/pci/devices/*/aer/correctable_ratelimit_burst
> /sys/bus/pci/devices/*/aer/correctable_ratelimit_interval_ms
> /sys/bus/pci/devices/*/aer/nonfatal_ratelimit_burst
> /sys/bus/pci/devices/*/aer/nonfatal_ratelimit_interval_ms
>
> The default values are ratelimit_burst=10, ratelimit_interval_ms=5000, so
> if we try to emit more than 10 messages in a 5 second period, some are
> suppressed.
>
> Update AER sysfs ABI filename to reflect the broader scope of AER sysfs
> attributes (e.g. stats and ratelimits).
>
> Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats ->
> sysfs-bus-pci-devices-aer
>
> Tested using aer-inject[1]. Configured correctable log ratelimit to 5.
> Sent 6 AER errors. Observed 5 errors logged while AER stats
> (cat /sys/bus/pci/devices/<dev>/aer_dev_correctable) shows 6.
>
> Disabled ratelimiting and sent 6 more AER errors. Observed all 6 errors
> logged and accounted in AER stats (12 total errors).
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/aer-inject.git
>
> [bhelgaas: note fatal errors are not ratelimited, "aer_report" ->
> "aer_info", replace ratelimit_log_enable toggle with *_ratelimit_interval_ms]
>
> Signed-off-by: Karolina Stolarek <karolina.stolarek@...cle.com>
> Signed-off-by: Jon Pan-Doh <pandoh@...gle.com>
> Signed-off-by: Bjorn Helgaas <bhelgaas@...gle.com>
> Link: https://patch.msgid.link/20250520215047.1350603-18-helgaas@kernel.org
> ---
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@...ux.intel.com>
> ...es-aer_stats => sysfs-bus-pci-devices-aer} | 44 ++++++++
> Documentation/PCI/pcieaer-howto.rst | 5 +-
> drivers/pci/pci-sysfs.c | 1 +
> drivers/pci/pci.h | 1 +
> drivers/pci/pcie/aer.c | 105 ++++++++++++++++++
> 5 files changed, 155 insertions(+), 1 deletion(-)
> rename Documentation/ABI/testing/{sysfs-bus-pci-devices-aer_stats => sysfs-bus-pci-devices-aer} (72%)
>
> diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats b/Documentation/ABI/testing/sysfs-bus-pci-devices-aer
> similarity index 72%
> rename from Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
> rename to Documentation/ABI/testing/sysfs-bus-pci-devices-aer
> index d1f67bb81d5d..5ed284523956 100644
> --- a/Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
> +++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-aer
> @@ -117,3 +117,47 @@ Date: July 2018
> KernelVersion: 4.19.0
> Contact: linux-pci@...r.kernel.org, rajatja@...gle.com
> Description: Total number of ERR_NONFATAL messages reported to rootport.
> +
> +PCIe AER ratelimits
> +-------------------
> +
> +These attributes show up under all the devices that are AER capable.
> +They represent configurable ratelimits of logs per error type.
> +
> +See Documentation/PCI/pcieaer-howto.rst for more info on ratelimits.
> +
> +What: /sys/bus/pci/devices/<dev>/aer/correctable_ratelimit_interval_ms
> +Date: May 2025
> +KernelVersion: 6.16.0
> +Contact: linux-pci@...r.kernel.org
> +Description: Writing 0 disables AER correctable error log ratelimiting.
> + Writing a positive value sets the ratelimit interval in ms.
> + Default is DEFAULT_RATELIMIT_INTERVAL (5000 ms).
> +
> +What: /sys/bus/pci/devices/<dev>/aer/correctable_ratelimit_burst
> +Date: May 2025
> +KernelVersion: 6.16.0
> +Contact: linux-pci@...r.kernel.org
> +Description: Ratelimit burst for correctable error logs. Writing a value
> + changes the number of errors (burst) allowed per interval
> + before ratelimiting. Reading gets the current ratelimit
> + burst. Default is DEFAULT_RATELIMIT_BURST (10).
> +
> +What: /sys/bus/pci/devices/<dev>/aer/nonfatal_ratelimit_interval_ms
> +Date: May 2025
> +KernelVersion: 6.16.0
> +Contact: linux-pci@...r.kernel.org
> +Description: Writing 0 disables AER non-fatal uncorrectable error log
> + ratelimiting. Writing a positive value sets the ratelimit
> + interval in ms. Default is DEFAULT_RATELIMIT_INTERVAL
> + (5000 ms).
> +
> +What: /sys/bus/pci/devices/<dev>/aer/nonfatal_ratelimit_burst
> +Date: May 2025
> +KernelVersion: 6.16.0
> +Contact: linux-pci@...r.kernel.org
> +Description: Ratelimit burst for non-fatal uncorrectable error logs.
> + Writing a value changes the number of errors (burst)
> + allowed per interval before ratelimiting. Reading gets the
> + current ratelimit burst. Default is DEFAULT_RATELIMIT_BURST
> + (10).
> diff --git a/Documentation/PCI/pcieaer-howto.rst b/Documentation/PCI/pcieaer-howto.rst
> index 6fb31516fff1..4b71e2f43ca7 100644
> --- a/Documentation/PCI/pcieaer-howto.rst
> +++ b/Documentation/PCI/pcieaer-howto.rst
> @@ -97,12 +97,15 @@ DPC errors, are not ratelimited.
> AER uses the default ratelimit of DEFAULT_RATELIMIT_BURST (10 events) over
> DEFAULT_RATELIMIT_INTERVAL (5 seconds).
>
> +Ratelimits are exposed in the form of sysfs attributes and configurable.
> +See Documentation/ABI/testing/sysfs-bus-pci-devices-aer.
> +
> AER Statistics / Counters
> -------------------------
>
> When PCIe AER errors are captured, the counters / statistics are also exposed
> in the form of sysfs attributes which are documented at
> -Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
> +Documentation/ABI/testing/sysfs-bus-pci-devices-aer.
>
> Developer Guide
> ===============
> diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
> index c6cda56ca52c..278de99b00ce 100644
> --- a/drivers/pci/pci-sysfs.c
> +++ b/drivers/pci/pci-sysfs.c
> @@ -1805,6 +1805,7 @@ const struct attribute_group *pci_dev_attr_groups[] = {
> &pcie_dev_attr_group,
> #ifdef CONFIG_PCIEAER
> &aer_stats_attr_group,
> + &aer_attr_group,
> #endif
> #ifdef CONFIG_PCIEASPM
> &aspm_ctrl_attr_group,
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 3023c68fe485..eca2812cfd25 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -965,6 +965,7 @@ void pci_no_aer(void);
> void pci_aer_init(struct pci_dev *dev);
> void pci_aer_exit(struct pci_dev *dev);
> extern const struct attribute_group aer_stats_attr_group;
> +extern const struct attribute_group aer_attr_group;
> void pci_aer_clear_fatal_status(struct pci_dev *dev);
> int pci_aer_clear_status(struct pci_dev *dev);
> int pci_aer_raw_clear_status(struct pci_dev *dev);
> diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
> index ebac126144fc..6c331695af58 100644
> --- a/drivers/pci/pcie/aer.c
> +++ b/drivers/pci/pcie/aer.c
> @@ -627,6 +627,111 @@ const struct attribute_group aer_stats_attr_group = {
> .is_visible = aer_stats_attrs_are_visible,
> };
>
> +/*
> + * Ratelimit interval
> + * <=0: disabled with ratelimit.interval = 0
> + * >0: enabled with ratelimit.interval in ms
> + */
> +#define aer_ratelimit_interval_attr(name, ratelimit) \
> + static ssize_t \
> + name##_show(struct device *dev, struct device_attribute *attr, \
> + char *buf) \
> + { \
> + struct pci_dev *pdev = to_pci_dev(dev); \
> + \
> + return sysfs_emit(buf, "%d\n", \
> + pdev->aer_info->ratelimit.interval); \
> + } \
> + \
> + static ssize_t \
> + name##_store(struct device *dev, struct device_attribute *attr, \
> + const char *buf, size_t count) \
> + { \
> + struct pci_dev *pdev = to_pci_dev(dev); \
> + int interval; \
> + \
> + if (!capable(CAP_SYS_ADMIN)) \
> + return -EPERM; \
> + \
> + if (kstrtoint(buf, 0, &interval) < 0) \
> + return -EINVAL; \
> + \
> + if (interval <= 0) \
> + interval = 0; \
> + else \
> + interval = msecs_to_jiffies(interval); \
> + \
> + pdev->aer_info->ratelimit.interval = interval; \
> + \
> + return count; \
> + } \
> + static DEVICE_ATTR_RW(name);
> +
> +#define aer_ratelimit_burst_attr(name, ratelimit) \
> + static ssize_t \
> + name##_show(struct device *dev, struct device_attribute *attr, \
> + char *buf) \
> + { \
> + struct pci_dev *pdev = to_pci_dev(dev); \
> + \
> + return sysfs_emit(buf, "%d\n", \
> + pdev->aer_info->ratelimit.burst); \
> + } \
> + \
> + static ssize_t \
> + name##_store(struct device *dev, struct device_attribute *attr, \
> + const char *buf, size_t count) \
> + { \
> + struct pci_dev *pdev = to_pci_dev(dev); \
> + int burst; \
> + \
> + if (!capable(CAP_SYS_ADMIN)) \
> + return -EPERM; \
> + \
> + if (kstrtoint(buf, 0, &burst) < 0) \
> + return -EINVAL; \
> + \
> + pdev->aer_info->ratelimit.burst = burst; \
> + \
> + return count; \
> + } \
> + static DEVICE_ATTR_RW(name);
> +
> +#define aer_ratelimit_attrs(name) \
> + aer_ratelimit_interval_attr(name##_ratelimit_interval_ms, \
> + name##_ratelimit) \
> + aer_ratelimit_burst_attr(name##_ratelimit_burst, \
> + name##_ratelimit)
> +
> +aer_ratelimit_attrs(correctable)
> +aer_ratelimit_attrs(nonfatal)
> +
> +static struct attribute *aer_attrs[] = {
> + &dev_attr_correctable_ratelimit_interval_ms.attr,
> + &dev_attr_correctable_ratelimit_burst.attr,
> + &dev_attr_nonfatal_ratelimit_interval_ms.attr,
> + &dev_attr_nonfatal_ratelimit_burst.attr,
> + NULL
> +};
> +
> +static umode_t aer_attrs_are_visible(struct kobject *kobj,
> + struct attribute *a, int n)
> +{
> + struct device *dev = kobj_to_dev(kobj);
> + struct pci_dev *pdev = to_pci_dev(dev);
> +
> + if (!pdev->aer_info)
> + return 0;
> +
> + return a->mode;
> +}
> +
> +const struct attribute_group aer_attr_group = {
> + .name = "aer",
> + .attrs = aer_attrs,
> + .is_visible = aer_attrs_are_visible,
> +};
> +
> static void pci_dev_aer_stats_incr(struct pci_dev *pdev,
> struct aer_err_info *info)
> {
--
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
Powered by blists - more mailing lists