linux-kernel - Re: Resurrecting the VM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 03 Mar 2015 19:35:12 +0100
From:	Vlastimil Babka <vbabka@...e.cz>
To:	Eric B Munson <emunson@...mai.com>, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, Peter Zijlstra <peterz@...radead.org>
CC:	Christoph Lameter <cl@...ux.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Hugh Dickins <hughd@...gle.com>, Mel Gorman <mgorman@...e.de>,
	Roland Dreier <roland@...nel.org>,
	Sean Hefty <sean.hefty@...el.com>,
	Hal Rosenstock <hal.rosenstock@...il.com>,
	Mike Marciniszyn <infinipath@...el.com>
Subject: Re: Resurrecting the VM_PINNED discussion

On 03/03/2015 06:41 PM, Eric B Munson wrote:> All,
>
> After LSF/MM last year Peter revived a patch set that would create
> infrastructure for pinning pages as opposed to simply locking them.
> AFAICT, there was no objection to the set, it just needed some help
> from the IB folks.
>
> Am I missing something about why it was never merged?  I ask because
> Akamai has bumped into the disconnect between the mlock manpage,
> Documentation/vm/unevictable-lru.txt, and reality WRT compaction and
> locking.  A group working in userspace read those sources and wrote a
> tool that mmaps many files read only and locked, munmapping them when
> they are no longer needed.  Locking is used because they cannot afford a
> major fault, but they are fine with minor faults.  This tends to
> fragment memory badly so when they started looking into using hugetlbfs
> (or anything requiring order > 0 allocations) they found they were not
> able to allocate the memory.  They were confused based on the referenced
> documentation as to why compaction would continually fail to yield
> appropriately sized contiguous areas when there was more than enough
> free memory.

So you are saying that mlocking (VM_LOCKED) prevents migration and thus
compaction to do its job? If that's true, I think it's a bug as it is AFAIK
supposed to work just fine.

> I would like to see the situation with VM_LOCKED cleared up, ideally the
> documentation would remain and reality adjusted to match and I think
> Peter's VM_PINNED set goes in the right direction for this goal.  What
> is missing and how can I help?

I don't think VM_PINNED would help you. In fact it is VM_PINNED that improves
accounting for the kind of locking (pinning) that *does* prevent page migration
(unlike mlocking)... quoting the patchset cover letter:

"These patches introduce VM_PINNED infrastructure, vma tracking of persistent
'pinned' page ranges. Pinned is anything that has a fixed phys address (as
required for say IO DMA engines) and thus cannot use the weaker VM_LOCKED. One
popular way to pin pages is through get_user_pages() but that not nessecarily
the only way."

> Thanks,
> Eric
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/