linux-kernel - Kernel panic - not syncing: System is deadlocked on memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <1FCA92FF-1FBC-41EA-B4DA-708849711758@csu.edu.cn>
Date: Sun, 11 May 2025 17:33:06 +0800
From: yinguoyu <yinguoyu@....edu.cn>
To: akpm@...ux-foundation.org <akpm@...ux-foundation.org>, 
 linux-mm@...ck.org <linux-mm@...ck.org>, 
 linux-kernel@...r.kernel.org <linux-kernel@...r.kernel.org>
Subject: Kernel panic - not syncing: System is deadlocked on memory

Hi,
When I test the Linux kernel (commit 38fec10eb60d687e30c8c6b5420d86e8149f7557), I encountered a kernel panic issue related to memory allocation, specifically in the alloc_pages_mpol function. The panic message indicates "System is deadlocked on memory," and the call trace points to __alloc_frozen_pages_noprof failing to allocate pages, leading to an OOM condition.
The error originates in alloc_pages_mpol, which calls policy_nodemask to determine the nodemask and nid for memory allocation. The subsequent call to __alloc_frozen_pages_noprof fails when the system runs out of memory, with no swap available and no killable processes. 
This crash can be triggered by executing the provided C reproducer multiple times. The reproducer performs a series of system calls (e.g., file creation, perf_event_open, io_submit, and filesystem mounts), which likely stress the memory management system and expose the bug.
This can be reproduced on:
HEAD commit: 
38fec10eb60d687e30c8c6b5420d86e8149f7557 
Console output, kernel config and a C reproducer as shown in the attachment.
Content of type "text/html" skipped

View attachment "console_output.txt" of type "text/plain" (3400 bytes)

View attachment "kernel_config.txt" of type "text/plain" (149638 bytes)

View attachment "C_repro.txt" of type "text/plain" (51561 bytes)