linux-kernel - Re: mmap: is default non-populating behavior stable?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Pine.LNX.4.64.0811051613400.21353@blonde.site>
Date:	Wed, 5 Nov 2008 16:42:22 +0000 (GMT)
From:	Hugh Dickins <hugh@...itas.com>
To:	"Eugene V. Lyubimkin" <jackyf.devel@...il.com>
cc:	Alan Cox <alan@...rguk.ukuu.org.uk>,
	Peter Zijlstra <peterz@...radead.org>,
	Chris Friesen <cfriesen@...tel.com>,
	Rik van Riel <riel@...hat.com>, linux-kernel@...r.kernel.org,
	linux-mm <linux-mm@...ck.org>
Subject: Re: mmap: is default non-populating behavior stable?

On Tue, 4 Nov 2008, Eugene V. Lyubimkin wrote:
> Alan Cox wrote:
> > 
> > I believe our behaviour is correct for mmap/mumap/truncate and it
> > certainly used to be and was tested.

Agreed.

> > 
> > At the point you do anything involving mremap (which is non posix) our
> > behaviour becomes rather bizarre.

Certainly mremap is non-POSIX, but I can't think of any way in which
it would interfere with Eugene's assumptions about population.

(Every year or so we do wonder whether to change an extending mremap
of a MAP_SHARED|MAP_ANONYMOUS object to extend the object itself instead
of just SIGBUSing on the extension: but I've so far remained conservative
about that, and Eugene appears to be thinking of more ordinary files.)

> 
> Thanks to all for answers. I have made the conclusion that doing "open() new
> file, truncate(<big size>), mmap(<the same big size>), write/read some memory
> pages" should not populate other, untouched by write/read pages (until
> MAP_POPULATE given), right?

That is a reasonable description of how the kernel tries and will always
try to handle it, approximately; but I don't think you can rely upon it
absolutely.

For a start, it depends on the filesystem: I believe that vfat, for
example, does not support the concept of sparse files (files with holes
in), so its truncate(<big size>) will allocate the whole of that big
size initially.

I'm not sure what you mean by "populate": in mm, as in MAP_POPULATE,
we're thinking of prefaulting pages into the user address space; but
you're probably thinking of whether the blocks are allocated on disk?

Prefaulting hole pages into the user address space may imply allocating
blocks on disk, or it may not: likely to depend on filesystem again.

>From time to time we toy with prefaulting adjacent pages when a fault
occurs (though IIRC tests have proved disappointing in the past): we'd
like to keep that option open, but it would go against your guidelines
above to some extent.

Hugh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/