linux-kernel - Re: remap_file

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 19 May 2014 13:34:05 -0400
From:	Kenny Simpson <theonetruekenny@...il.com>
To:	"Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
Cc:	Michal Hocko <mhocko@...e.cz>, linux-kernel@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Dave Jones <davej@...hat.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Armin Rigo <arigo@...es.org>
Subject: Re: remap_file_pages() use

For the circular buffer case, yes I could make some temp file under
/dev/shm, unlink it, mmap() it multiple times, etc... its just makes
it a little more hairy.
https://lkml.org/lkml/2011/3/19/94

For the other cases I had used the remapping to have more of a sliding
window over a disk-backed file where I also was using aliasing to
eliminate the corner cases of hitting the end of a window and needing
to split records due to crossing boundaries, etc..  These were done
for other projects in the past, but I am no longer on those projects
(or with some of the companies), so I can't as easily double check the
code or make changes.
Being able to replace mappings in-place was logically simpler than
doing mmap() over the old mappings.  I was also under the impression
that it should have less pressure on the mm_sem - I'm pretty sure I
did profile it many years ago (>5), but many things have changed since
then so maybe this is not a big deal any more.  This use case was more
performance-critical than the one-time setup of the anonymous circular
buffer as it was more like a transaction log/journal.

All that said, if remap_file_pages were to go away as of a mainline
kernel this year, the projects I had worked on would probably not be
impacted for many years and would probably have been retired/rewritten
before they would ever see the change (unless some Linux vendor in
N.C. backported its removal).  Since valgrind doesn't support it, I
tend to write fallback/debug variants anyway.

-Kenny

On Mon, May 19, 2014 at 10:35 AM, Kirill A. Shutemov
<kirill.shutemov@...ux.intel.com> wrote:
> Michal Hocko wrote:
>> [CCing Kirill and other people involved]
>>
>> On Sun 18-05-14 00:03:28, Kenny Simpson wrote:
>> > I saw that remap_file_pages() was possibly going away to be replaced
>> > by some emulation. I've used this call in several projects over the
>> > years mostly as a way of mapping multiple virtual memory pages to
>> > alias the same private or shared memory region (to do things like
>> > circular buffers). mmap()
>> > in the case of anonymous memory doesn't work as well since there is
>> > not a file descriptor to reference.
>> >
>> > Would this sort of thing be supported in the emulation, or should I be
>> > planning on reimplementing/rewriting some things?
>
> From functional POV, emulation *should* be identical to original
> remap_file_pages(), but slower. It would be nice, if you test it early.
>
> It's not clear yet how long emulation will be there.
> Is there a reason why you can't use fd from shared memory (shm_open() or
> direct open() on /dev/shm/xxx)?
>
> --
>  Kirill A. Shutemov
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/