linux-kernel - Re: mlock() confusing 1 half of system RAM limit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180601130524.GB15278@dhcp22.suse.cz>
Date:   Fri, 1 Jun 2018 15:05:24 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Alex Richman <alex.r@...abs.co.uk>
Cc:     linux-man@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: mlock() confusing 1 half of system RAM limit

On Fri 01-06-18 13:26:59, Alex Richman wrote:
> I am using a shm MAP_SHARED, along these lines:
> > shm_fd = shm_open(handle, (O_RDWR | O_CREAT), (S_IRWXU | S_IRWXG |
> S_IRWXO));
> > ftruncate(shm_fd, channel->sled_size)
> > channel->sled = mmap(NULL, channel->sled_size, (PROT_READ | PROT_WRITE),
> > (MAP_SHARED | MAP_NORESERVE), shm_fd, 0);
> > mlock(channel->sled, channel->sled_size) /* Fails with ENOMEM. */
> 
> But shmmax is unlimited on my box:
> # sysctl -a | grep shm
> kernel.shm_next_id = -1
> kernel.shm_rmid_forced = 0
> kernel.shmall = 18446744073692774399
> kernel.shmmax = 18446744073692774399
> kernel.shmmni = 4096
> 
> Any ideas?

shm_open uses tmpfs/shmem under the cover and that has the internal
limit as explained above.
-- 
Michal Hocko
SUSE Labs