linux-kernel - whole system lock-up on low memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [day] [month] [year] [list]

Date:	Mon, 11 May 2009 01:45:52 +0200
From:	Ulrich Lukas <stellplatz-nr.13a@...enparkplatz.de>
To:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: whole system lock-up on low memory

Hi,

vanilla linux 2.6.29.3, AMD64, tested on openSUSE 11.1 and Kubuntu 9.04.

I observe the following behaviour:

When any user application (non-kernel, non-root) consumes all the
available system memory, the system freezes completely instead of any
application being killed by the oom-killer.

The mouse pointer stalls, and there is a seemingly endless loop of
hard-disc access, even when no swap space on any harddisc is activated.

IMO, this is severe, because any application can practically crash the
system (e.g. in case of a memory-leak), causing data loss in case of
unsaved data.

To reproduce this, I've attached a small C++ utility
(compiles with g++ memory_overcommit.cc -o memory_overcommit.bin) which
allocates chunks of memory of user defined size.

On the Ubuntu system, the system freeze can be observed with swap
enabled on a cryptographic swap partition (dm-crypt; /etc/crypttab).
With openSUSE the lock-up also occurs with deactivated swap.
(swapoff -a).

Tested from a KDE terminal window as regular urser, the
steps to reproduce are:

* use cryptographic swap partition or disable swap
  (maybe also reproducible with normal swap; but apparently not on Suse)

* compile and invoke the attached code:
  ./memory_overcommit.bin

* enter a number (in MiB) of memory that is slightly smaller than the
available memory and press "enter" key once.

* enter a smaller number (minimum 1 MiB) and confirm again, do the same
again,..., successively approaching the limit of available memory with
smaller chunks.

* finally, when most of the memory/buffers/cache are used-up, the system
becomes unresponsive and constant, heavy harddisk-access commences.

Sometimes, killing the X-server or shutting down via hotkeys works after
several minutes of waiting, but this is not consistent.

I've reported this issue on the Ubuntu bug tracker before:
https://bugs.launchpad.net/ubuntu/+bug/283420

but as stated above, I also had the problem on another system.

I'd be glad if I could help if you need further information.

View attachment "memory_overcommit.cc" of type "text/x-c++src" (3795 bytes)