linux-kernel - Re: [PATCH] ghes: Track number of recovered hardware errors

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <p2iytcdfvgm74zif6ihd7gs4kuaeza4b4p52cr5ya4upabiome@kr3yy7fjznwe>
Date: Tue, 15 Jul 2025 10:25:46 -0700
From: Breno Leitao <leitao@...ian.org>
To: Shuai Xue <xueshuai@...ux.alibaba.com>
Cc: Borislav Petkov <bp@...en8.de>, Alexander Graf <graf@...zon.com>, 
	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>, Peter Gonda <pgonda@...gle.com>, 
	"Luck, Tony" <tony.luck@...el.com>, "Rafael J. Wysocki" <rafael@...nel.org>, 
	Len Brown <lenb@...nel.org>, James Morse <james.morse@....com>, 
	"Moore, Robert" <robert.moore@...el.com>, "linux-acpi@...r.kernel.org" <linux-acpi@...r.kernel.org>, 
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>, "acpica-devel@...ts.linux.dev" <acpica-devel@...ts.linux.dev>, 
	"kernel-team@...a.com" <kernel-team@...a.com>
Subject: Re: [PATCH] ghes: Track number of recovered hardware errors

Hello Shuai,

On Tue, Jul 15, 2025 at 09:46:03PM +0800, Shuai Xue wrote:
> > It would be really good to sync with other cloud providers here so that we can
> > do this one solution which fits all. Lemme CC some other folks I know who do
> > cloud gunk and leave the whole mail for their pleasure.
> > 
> > Newly CCed folks, you know how to find the whole discussion. :-)
> > 
> > Thx.
> 
> 
> For the purpose of counting, how about using the cmdline of rasdaemon?

How do you manage it at a large fleet of hosts? Do you have rasdaemon
logging always and how do you correlate with kernel crashes? At Meta, we
have an a "clues" tag for each crash, and one of the tags is Machine
Check Exception (MCE), which is parsed from dmesg right now (with the
regexp I shared earlier).

My plan with this patch is to have a counter for hardware errors that
would be exposed to the crashdump. So, post-morten analyzes tooling can
easily query if there are hardware errors and query RAS information in
the right databases, in case it seems a smoking gun.

Do you have any experience with this type of automatic correlation?

Thanks for your insights,
--breno