linux-kernel - Re: mlx5: CX7: fw_tracer: crash at mlx5_tracer_print

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <e9abc694-27f2-4064-873c-76859573a921@nvidia.com>
Date: Wed, 15 Oct 2025 12:13:29 +0300
From: Moshe Shemesh <moshe@...dia.com>
To: Breno Leitao <leitao@...ian.org>, <saeedm@...dia.com>,
	<itayavr@...dia.com>
CC: <netdev@...r.kernel.org>, <linux-kernel@...r.kernel.org>,
	<dcostantino@...a.com>, <kuba@...nel.org>
Subject: Re: mlx5: CX7: fw_tracer: crash at mlx5_tracer_print_trace()



On 10/9/2025 3:42 PM, Breno Leitao wrote:
> Hello,
> 
> I am seeing a crash in some production host in function
> mlx5_tracer_print_trace() that sprintf a string (%s) pointing to value
> that doesn't seem to be addressable. I am seeing this on 6.13, but,
> looking at the upstream code, the function is the same.
> 
> Unfortunately I am not able to reproduce this on upstream kernel easily.
> Host is running ConnectX-7.
> 
> Here is the quick stack of the problem:
> 
> 	Unable to handle kernel paging request at virtual address 00000000213afe58
> 
> 	#0  string_nocheck(buf=0xffff8002a11af909[vmap stack: 1315725 (kworker/u576:1) +0xf909], end=0xffff8002a11afae0[vmap stack: 1315725 (kworker/u576:1) +0xfae0], s=0x213afe59, len=0) (lib/vsprintf.c:646:12)
> 	#1  string(end=0xffff8002a11afae0[vmap stack: 1315725 (kworker/u576:1) +0xfae0], s=0x213afe58) (lib/vsprintf.c:728:9)
> 	#2  vsnprintf(buf=0xffff8002a11af8e0[vmap stack: 1315725 (kworker/u576:1) +0xf8e0], fmt=0xffff10006cd4950a, end=0xffff8002a11afae0[vmap stack: 1315725 (kworker/u576:1) +0xfae0], str=0xffff8002a11af909[vmap stack: 1315725 (kworker/u576:1) +0xf909], old_fmt=0xffff10006cd49508) (lib/vsprintf.c:2848:10)
> 	#3  snprintf (lib/vsprintf.c:2983:6)
> 
> Looking further, I found this code:
> 
>          snprintf(tmp, sizeof(tmp), str_frmt->string,
>                   str_frmt->params[0],
>                   str_frmt->params[1],
>                   str_frmt->params[2],
>                   str_frmt->params[3],
>                   str_frmt->params[4],
>                   str_frmt->params[5],
>                   str_frmt->params[6]);
> 
> 
> and the str_frmt has the following content:
> 
> 	*(struct tracer_string_format *)0xffff100026547260 = {
> 	.string = (char *)0xffff10006cd494df = "PCA 9655E init, failed to verify command %s, failed %d",
> 	.params = (int [7]){ 557514328, 3 },
> 	.num_of_params = (int)2,
> 	.last_param_num = (int)2,
> 	.event_id = (u8)3,
> 	.tmsn = (u32)5201,
> 	.hlist = (struct hlist_node){
> 		.next = (struct hlist_node *)0xffff0009f63ce078,
> 		.pprev = (struct hlist_node **)0xffff0004123ec8d8,
> 	},
> 	.list = (struct list_head){
> 		.next = (struct list_head *)0xdead000000000100,
> 		.prev = (struct list_head *)0xdead000000000122,
> 	},
> 	.timestamp = (u32)22,
> 	.lost = (bool)0,
> 	}
> 
> 
> My understanding that we are printf %s with params[0], which is 557514328 (aka
> 0x213afe58). So, sprintf is trying to access the content of 0x213afe58, which
> is invalid, and crash.
> 
> Is this a known issue?
> 

Not a known issue, not expected, thanks for reporting.
We will send patch to protect from such crash.
Please send FW version it was detected on.

Thanks,
Moshe.

> Thanks
> --breno
>