[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <31bd945c-b1b5-4899-b089-bc7f5b2e5e65@amd.com>
Date: Mon, 6 Oct 2025 16:07:46 -0500
From: "Bowman, Terry" <terry.bowman@....com>
To: "Cheatham, Benjamin" <benjamin.cheatham@....com>
Cc: linux-kernel@...r.kernel.org, linux-pci@...r.kernel.org,
dave@...olabs.net, jonathan.cameron@...wei.com, dave.jiang@...el.com,
alison.schofield@...el.com, dan.j.williams@...el.com, bhelgaas@...gle.com,
shiju.jose@...wei.com, ming.li@...omail.com,
Smita.KoralahalliChannabasappa@....com, rrichter@....com,
dan.carpenter@...aro.org, PradeepVineshReddy.Kodamati@....com,
lukas@...ner.de, sathyanarayanan.kuppuswamy@...ux.intel.com,
linux-cxl@...r.kernel.org, alucerop@....com, ira.weiny@...el.com
Subject: Re: [PATCH v12 17/25] cxl/pci: Introduce CXL Endpoint protocol error
handlers
On 10/3/2025 3:12 PM, Cheatham, Benjamin wrote:
> On 9/25/2025 5:34 PM, Terry Bowman wrote:
>> CXL Endpoint protocol errors are currently handled using PCI error
>> handlers. The CXL Endpoint requires CXL specific handling in the case of
>> uncorrectable error (UCE) handling not provided by the PCI handlers.
>>
>> Add CXL specific handlers for CXL Endpoints. Rename the existing
>> cxl_error_handlers to be pci_error_handlers to more correctly indicate
>> the error type and follow naming consistency.
>>
>> The PCI handlers will be called if the CXL device is not trained for
>> alternate protocol (CXL). Update the CXL Endpoint PCI handlers to call the
>> CXL UCE handlers.
>>
>> The existing EP UCE handler includes checks for various results. These are
>> no longer needed because CXL UCE recovery will not be attempted. Implement
>> cxl_handle_ras() to return PCI_ERS_RESULT_NONE or PCI_ERS_RESULT_PANIC. The
>> CXL UCE handler is called by cxl_do_recovery() that acts on the return
>> value. In the case of the PCI handler path, call panic() if the result is
>> PCI_ERS_RESULT_PANIC.
>>
>> Signed-off-by: Terry Bowman <terry.bowman@....com>
>> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@...ux.intel.com>
>>
>> ---
>>
>> Changes in v11->v12:
>> - None
>>
>> Changes in v10->v11:
>> - cxl_error_detected() - Change handlers' scoped_guard() to guard() (Jonathan)
>> - cxl_error_detected() - Remove extra line (Shiju)
>> - Changes moved to core/ras.c (Terry)
>> - cxl_error_detected(), remove 'ue' and return with function call. (Jonathan)
>> - Remove extra space in documentation for PCI_ERS_RESULT_PANIC definition
>> - Move #include "pci.h from cxl.h to core.h (Terry)
>> - Remove unnecessary includes of cxl.h and core.h in mem.c (Terry)
>> ---
>> drivers/cxl/core/core.h | 17 +++++++
>> drivers/cxl/core/ras.c | 110 +++++++++++++++++++---------------------
>> drivers/cxl/cxlpci.h | 15 ------
>> drivers/cxl/pci.c | 9 ++--
>> 4 files changed, 75 insertions(+), 76 deletions(-)
>>
>> diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
>> index 8c51a2631716..74c64d458f12 100644
>> --- a/drivers/cxl/core/core.h
>> +++ b/drivers/cxl/core/core.h
>> @@ -6,6 +6,7 @@
>>
>> #include <cxl/mailbox.h>
>> #include <linux/rwsem.h>
>> +#include <linux/pci.h>
>>
>> extern const struct device_type cxl_nvdimm_bridge_type;
>> extern const struct device_type cxl_nvdimm_type;
>> @@ -150,6 +151,11 @@ void cxl_ras_exit(void);
>> void cxl_switch_port_init_ras(struct cxl_port *port);
>> void cxl_endpoint_port_init_ras(struct cxl_port *ep);
>> void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host);
>> +pci_ers_result_t pci_error_detected(struct pci_dev *pdev,
>> + pci_channel_state_t error);
>> +void pci_cor_error_detected(struct pci_dev *pdev);
>> +void cxl_cor_error_detected(struct device *dev);
>> +pci_ers_result_t cxl_error_detected(struct device *dev);
>> #else
>> static inline int cxl_ras_init(void)
>> {
>> @@ -163,6 +169,17 @@ static inline void cxl_switch_port_init_ras(struct cxl_port *port) { }
>> static inline void cxl_endpoint_port_init_ras(struct cxl_port *ep) { }
>> static inline void cxl_dport_init_ras_reporting(struct cxl_dport *dport,
>> struct device *host) { }
>> +static inline pci_ers_result_t pci_error_detected(struct pci_dev *pdev,
>> + pci_channel_state_t error)
>> +{
>> + return PCI_ERS_RESULT_NONE;
>> +}
>> +static inline void pci_cor_error_detected(struct pci_dev *pdev) { }
>> +static inline void cxl_cor_error_detected(struct device *dev) { }
>> +static inline pci_ers_result_t cxl_error_detected(struct device *dev)
>> +{
>> + return PCI_ERS_RESULT_NONE;
> My understanding is this only occurs for uncorrectable errors, so should this be upgraded to
> a PCI_ERS_RESULT_PANIC? If uncorrectable errors == system panic, I would expect that to be the
> case even if we don't have the code to handle the error built.
>
> I guess it's really a question of how safe you want to be. Is it ok to let uncorrectable errors
> propagate when the support is missing, or do we always panic regardless of handling code?
Here the CONFIG_CXL_RAS Kconfig is disabled and these function stubs allow the linker to complete
the build.PCI_ERS_RESULT_PANIC isn't returned because it implies handling but handling is
disabled through unset CONFIG_CXL_RAS. If CONFIG_CXL_RAS is disabled then the interrupts
and CXL RAS logic should be disabled.
>> +}
>> #endif // CONFIG_CXL_RAS
>>
>> int cxl_gpf_port_setup(struct cxl_dport *dport);
>> diff --git a/drivers/cxl/core/ras.c b/drivers/cxl/core/ras.c
>> index 14a434bd68f0..39472d82d586 100644
>> --- a/drivers/cxl/core/ras.c
>> +++ b/drivers/cxl/core/ras.c
>> @@ -129,7 +129,7 @@ void cxl_ras_exit(void)
>> }
>>
>> static void cxl_handle_cor_ras(struct device *dev, u64 serial, void __iomem *ras_base);
>> -static bool cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_base);
>> +static pci_ers_result_t cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_base);
>>
>> #ifdef CONFIG_CXL_RCH_RAS
>> static void cxl_dport_map_rch_aer(struct cxl_dport *dport)
>> @@ -371,7 +371,7 @@ static void header_log_copy(void __iomem *ras_base, u32 *log)
>> * Log the state of the RAS status registers and prepare them to log the
>> * next error status. Return 1 if reset needed.
>> */
>> -static bool cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_base)
>> +static pci_ers_result_t cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_base)
>> {
>> u32 hl[CXL_HEADERLOG_SIZE_U32];
>> void __iomem *addr;
>> @@ -380,13 +380,13 @@ static bool cxl_handle_ras(struct device *dev, u64 serial, void __iomem *ras_bas
>>
>> if (!ras_base) {
>> dev_warn_once(dev, "CXL RAS register block is not mapped");
>> - return false;
>> + return PCI_ERS_RESULT_NONE;
> Same idea as above. I would assume since we can't tell the severity of the error we would
> just treat it as the worst case scenario.
>
The RAS UCE status needs to be read and verified as UCE before handling with a panic.
Terry
Powered by blists - more mailing lists