linux-kernel - Re: [PATCH] ghes: Track number of recovered hardware errors

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <564e0deb-73f0-449f-9d0f-181311d3a348@linux.alibaba.com>
Date: Wed, 16 Jul 2025 10:05:27 +0800
From: Shuai Xue <xueshuai@...ux.alibaba.com>
To: Borislav Petkov <bp@...en8.de>, mchehab+huawei@...nel.org
Cc: Breno Leitao <leitao@...ian.org>, Alexander Graf <graf@...zon.com>,
 Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
 Peter Gonda <pgonda@...gle.com>, "Luck, Tony" <tony.luck@...el.com>,
 "Rafael J. Wysocki" <rafael@...nel.org>, Len Brown <lenb@...nel.org>,
 James Morse <james.morse@....com>, "Moore, Robert" <robert.moore@...el.com>,
 "linux-acpi@...r.kernel.org" <linux-acpi@...r.kernel.org>,
 "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
 "acpica-devel@...ts.linux.dev" <acpica-devel@...ts.linux.dev>,
 "kernel-team@...a.com" <kernel-team@...a.com>
Subject: Re: [PATCH] ghes: Track number of recovered hardware errors

在 2025/7/15 23:09, Borislav Petkov 写道:
> On Tue, Jul 15, 2025 at 09:46:03PM +0800, Shuai Xue wrote:
>> For the purpose of counting, how about using the cmdline of rasdaemon?
> 
> That would mean you have to run rasdaemon on those machines before they
> explode and then carve out the rasdaemon db from the coredump (this is
> post-mortem analysis).

Rasdaemon is a userspace tool that will collect all hardware error 
events reported by the Linux Kernel from several sources (EDAC, MCE, 
PCI, ...) into one common framework. And it has been a standard tools
in Alibaba. As far as I know, twitter also use Rasdaemon in its production.

> 
> I would love for rasdaemon to log over the network and then other tools can
> query those centralized logs but that has its own challenges...
> 

I also prefer collecting rasdaemon data in a centralized data center, as 
this is more beneficial for using big data analytics to analyze and 
predict errors. At the same time, the centralized side also uses 
rasdaemon logs as one of the references for machine operations and 
maintenance.

As for rasdaemon itself, it is just a single-node event collector and 
database, although it does also print logs. In practice, we use SLS [1] 
to collect rasdaemon text logs from individual nodes and parse them on 
the central side.

Thanks.
Shuai

[1]https://www.alibabacloud.com/help/en/sls/getting-started
[2]https://blog.x.com/engineering/en_us/topics/infrastructure/2023/how-twitter-uses-rasdaemon-for-hardware-reliability