linux-kernel - Re: [patch] mm, madvise: fail with ENOMEM when splitting vma will hit max_map

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 25 Jan 2017 16:00:13 +0530
From:   Anshuman Khandual <khandual@...ux.vnet.ibm.com>
To:     David Rientjes <rientjes@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>
Cc:     Jonathan Corbet <corbet@....net>,
        Johannes Weiner <hannes@...xchg.org>, mtk.manpages@...il.com,
        Jerome Marchand <jmarchan@...hat.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        linux-man@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org
Subject: Re: [patch] mm, madvise: fail with ENOMEM when splitting vma will hit
 max_map_count

On 01/25/2017 04:02 AM, David Rientjes wrote:
> If madvise(2) advice will result in the underlying vma being split and
> the number of areas mapped by the process will exceed
> /proc/sys/vm/max_map_count as a result, return ENOMEM instead of EAGAIN.
> 
> EAGAIN is returned by madvise(2) when a kernel resource, such as slab,
> is temporarily unavailable.  It indicates that userspace should retry the
> advice in the near future.  This is important for advice such as
> MADV_DONTNEED which is often used by malloc implementations to free
> memory back to the system: we really do want to free memory back when
> madvise(2) returns EAGAIN because slab allocations (for vmas, anon_vmas,
> or mempolicies) cannot be allocated.
> 
> Encountering /proc/sys/vm/max_map_count is not a temporary failure,
> however, so return ENOMEM to indicate this is a more serious issue.  A
> followup patch to the man page will specify this behavior.

But in the due course there might be other changes in number of VMAs of
the process because of unmap() or merge() which could reduce the total
number of VMAs and hence this condition may not exist afterwards. In
that case EAGAIN still makes sense.