linux-kernel - [PATCH 0/2] Fixes for hugetlbfs-related problems on shared memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 27 May 2009 12:12:27 +0100
From:	Mel Gorman <mel@....ul.ie>
To:	Ingo Molnar <mingo@...e.hu>,
	Andrew Morton <akpm@...ux-foundation.org>, stable@...nel.org,
	Linux Memory Management List <linux-mm@...ck.org>
Cc:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Hugh Dickins <hugh.dickins@...cali.co.uk>,
	Lee Schermerhorn <Lee.Schermerhorn@...com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	starlight@...nacle.cx, Eric B Munson <ebmunson@...ibm.com>,
	Adam Litke <agl@...ibm.com>,
	Andy Whitcroft <apw@...onical.com>, wli@...ementarian.org
Subject: [PATCH 0/2] Fixes for hugetlbfs-related problems on shared memory

The following two patches are required to fix problems reported by
starlight@...nacle.cx. The tests cases both involve two processes interacting
with shared memory segments backed by hugetlbfs.

Patch 1 fixes an x86-specific problem where regions sharing page tables
are not being reference counted properly. The page tables get freed early
resulting in bad PMD messages printed to the kernel log and the hugetlb
counters getting corrupted. Strictly speaking, this affects mainline but
the problem is masked by UNEVITABLE_LRU as it never leaves VM_LOCKED set for
hugetlbfs-backed mapping. This does affect the stable branch of 2.6.27 and
distributions based on that kernel such as SLES 11. This patch is required
for 2.6.27-stable and while it is optional for mainline, it should be merged
so that the stable branch does not contain patches that are not in mainline.

Patch 2 fixes a general hugetlbfs problem where it is using VM_SHARED instead
of VM_MAYSHARE to detect if the mapping was MAP_SHARED or MAP_PRIVATE. This
causes hugetlbfs to attempt reserving more pages than is required for
MAP_SHARED and mmap() fails when it should succeed. This patch is needed
for 2.6.30 and -stable. It rejects against 2.6.27.24 but the reject is
trivially resolved by changing the last VM_SHARED in hugetlb_reserve_pages()
to VM_MAYSHARE.

Starlight, if you are still watching, can you reconfirm that this patches
fix the problems you were having?

 arch/x86/mm/hugetlbpage.c |    6 +++++-
 mm/hugetlb.c              |   26 +++++++++++++-------------
 2 files changed, 18 insertions(+), 14 deletions(-)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/