linux-kernel - Large-region mmap unexpectedly fails with ENOMEM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:	Sun, 12 Dec 2010 03:13:14 +0100 (CET)
From:	Jan Engelhardt <jengelh@...ozas.de>
To:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Large-region mmap unexpectedly fails with ENOMEM

Greetings.


When working with large files, such as git has to when dealing with 
packs, I noticed that mmap fails when the requested length of the 
mapping is approaching the size of physical memory (no swap configured). 
It is impossible to — even just read-only — call mmap on a file or 
anonymous object and requesting a region despite 64-bit architectures 
having the address space for it.

This looks like a bug to me.
I presume that mapping a, say 2 GB file on a system with 2 GB RAM does 
not require 2 GB just for the PTEs.
Observed on 2.6.33, 2.6.36.

---
$ ls -log devqa.dsk 
-rw-r----- 1  8589934592 Nov 30 02:58 devqa.dsk
$ ./mmap devqa.dsk $[1048576*2048]
sizeof(size_t) = 8
devqa.dsk for 2147483648
mmap: Cannot allocate memory
0xffffffffffffffff

$ free
             total    used    free  shared buffers  cached
Mem:       2051340  113076 1938264       0   10928   67776
-/+ buffers/cache:   34372 2016968
Swap:            0       0       0

/proc/meminfo:
MemTotal:        2051340 kB
MemFree:         1938264 kB
Buffers:           10936 kB
Cached:            58848 kB
SwapCached:            0 kB
Active:            35560 kB
Inactive:          45696 kB
Active(anon):      11828 kB
Inactive(anon):        0 kB
Active(file):      23732 kB
Inactive(file):    45696 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 8 kB
Writeback:             0 kB
AnonPages:         11468 kB
Mapped:             8964 kB
Shmem:               360 kB
Slab:              16976 kB
SReclaimable:       8932 kB
SUnreclaim:         8044 kB
KernelStack:        1032 kB
PageTables:         2120 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1025668 kB
Committed_AS:      82680 kB
VmallocTotal:   34359738367 kB
VmallocUsed:        2672 kB
VmallocChunk:   34359730812 kB
DirectMap4k:        8128 kB
DirectMap2M:     2088960 kB
---
#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/types.h>

int main(int argc, const char **argv)
{
	int fd = open(argv[1], O_RDONLY);
	if (fd < 0)
		abort();
	size_t len = strtoull(argv[2], NULL, 0);
	printf("sizeof(size_t) = %zu\n", sizeof(size_t));
	if (sizeof(size_t) < 8)
		abort();
	printf("%s for %zu\n", argv[1], len);
	void *x = mmap(NULL, len, PROT_READ, MAP_SHARED, fd, 0);
	if (x == (void *)-1)
		perror("mmap");
	printf("%p\n", x);
	return 0;
}
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/