linux-kernel - Re: Resurrecting the VM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 3 Mar 2015 16:52:58 -0500
From:	Eric B Munson <emunson@...mai.com>
To:	Vlastimil Babka <vbabka@...e.cz>
Cc:	linux-mm@...ck.org, linux-kernel@...r.kernel.org,
	Peter Zijlstra <peterz@...radead.org>,
	Christoph Lameter <cl@...ux.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Hugh Dickins <hughd@...gle.com>, Mel Gorman <mgorman@...e.de>,
	Roland Dreier <roland@...nel.org>,
	Sean Hefty <sean.hefty@...el.com>,
	Hal Rosenstock <hal.rosenstock@...il.com>,
	Mike Marciniszyn <infinipath@...el.com>
Subject: Re: Resurrecting the VM_PINNED discussion

On Tue, 03 Mar 2015, Eric B Munson wrote:

> On Tue, 03 Mar 2015, Vlastimil Babka wrote:
> 
> > On 03/03/2015 07:45 PM, Eric B Munson wrote:
> > > On Tue, 03 Mar 2015, Vlastimil Babka wrote:
> > > 
> > >> On 03/03/2015 06:41 PM, Eric B Munson wrote:> All,
> > >> >
> > >> > After LSF/MM last year Peter revived a patch set that would create
> > >> > infrastructure for pinning pages as opposed to simply locking them.
> > >> > AFAICT, there was no objection to the set, it just needed some help
> > >> > from the IB folks.
> > >> >
> > >> > Am I missing something about why it was never merged?  I ask because
> > >> > Akamai has bumped into the disconnect between the mlock manpage,
> > >> > Documentation/vm/unevictable-lru.txt, and reality WRT compaction and
> > >> > locking.  A group working in userspace read those sources and wrote a
> > >> > tool that mmaps many files read only and locked, munmapping them when
> > >> > they are no longer needed.  Locking is used because they cannot afford a
> > >> > major fault, but they are fine with minor faults.  This tends to
> > >> > fragment memory badly so when they started looking into using hugetlbfs
> > >> > (or anything requiring order > 0 allocations) they found they were not
> > >> > able to allocate the memory.  They were confused based on the referenced
> > >> > documentation as to why compaction would continually fail to yield
> > >> > appropriately sized contiguous areas when there was more than enough
> > >> > free memory.
> > >> 
> > >> So you are saying that mlocking (VM_LOCKED) prevents migration and thus
> > >> compaction to do its job? If that's true, I think it's a bug as it is AFAIK
> > >> supposed to work just fine.
> > > 
> > > Agreed.  But as has been discussed in the threads around the VM_PINNED
> > > work, there are people that are relying on the fact that VM_LOCKED
> > > promises no minor faults.  Which is why the behavoir has remained.
> > 
> > At least in the VM_PINNED thread after last lsf/mm, I don't see this mentioned.
> > I found no references to mlocking in compaction.c, and in migrate.c there's just
> > mlock_migrate_page() with comment:
> > 
> > /*
> >  * mlock_migrate_page - called only from migrate_page_copy() to
> >  * migrate the Mlocked page flag; update statistics.
> >  */
> > 
> > It also passes TTU_IGNORE_MLOCK to try_to_unmap(). So what am I missing? Where
> > is this restriction?
> > 
> 
> I spent quite some time looking for it as well, it is in vmscan.c
> 
> int __isolate_lru_page(struct page *page, isolate_mode_t mode)
> {
> ...
>         /* Compaction should not handle unevictable pages but CMA can do so */
>         if (PageUnevictable(page) && !(mode & ISOLATE_UNEVICTABLE))
>                 return ret;
> ...
> 
> 

And that demonstrates that I haven't spent enough time with this code,
that isn't the restriction because when this is called from compaction.c
the mode is set to ISOLATE_UNEVICTABLE.  So back to reading the code.

Eric

Download attachment "signature.asc" of type "application/pgp-signature" (820 bytes)