linux-kernel - Re: Hard system lock-ups when using encrypted swap and RAM is exhausted

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <804601778974c504d42f4423d335a94d@va1der.ca>
Date: Thu, 27 Nov 2025 20:51:23 -0400
From: Kurt Fitzner <kurt_cryptsetup@...der.ca>
To: Mikulas Patocka <mpatocka@...hat.com>
Cc: Aaron Rainbolt <arraybolt3@...il.com>, Milan Broz <gmazyland@...il.com>,
 linux-mm@...ck.org, cryptsetup@...ts.linux.dev, dm-devel@...ts.linux.dev,
 linux-kernel@...r.kernel.org, adrelanos@...nix.org
Subject: Re: Hard system lock-ups when using encrypted swap and RAM is
 exhausted

On 2025-11-27 13:54, Mikulas Patocka wrote:

> Encrypted swap file is not supposed to work.

Do you have a reference for this?  The concept of encrypted swap files 
has been a valid workflow for a very long time.

> So, this is what happened to you - the machine runs out of memory, it
> needs to swap out some pages, dm-crypt encrypts the pages and generates
> write bios, the write bios are directed to the loop device, the loop
> device directs them to the filesystem, the filesystem attempts to 
> allocate
> more memory => deadlock.

If it's the filesystem trying to allocate memory on writes to a swap 
file that is causing a memory allocation/swap race, then any write to a 
swap file would engender the same result, regardless of encryption. The 
encryption layer is redundant under the failure mode you propose.

I can confirm I have put kernels up to and including 6.14 under heavy 
memory stress and have never encountered anything that feels like a 
memory allocation race.  All my systems have encrypted swap files.

I can't speak toward later kernels, or any bugs that may or may not be 
presesnt, but I know of nothing to suggest that encrypted swap files 
remain anything other than an intended feature.

     Kurt