linux-kernel - [RFC] Re: Costless huge virtual memory? /dev/same, /dev/null?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170413094200.b4lftvumqt4g36hz@var.youpi.perso.aquilenet.fr>
Date:   Thu, 13 Apr 2017 11:42:00 +0200
From:   Samuel Thibault <samuel.thibault@...-lyon.org>
To:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Arnd Bergmann <arnd@...db.de>
Cc:     linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: [RFC] Re: Costless huge virtual memory? /dev/same, /dev/null?

Hello,

More than one year passed without any activity :)

I have attached a proposed patch for discussion.

Samuel

Samuel Thibault, on lun. 29 févr. 2016 17:28:35 +0100, wrote:
> I'm wondering whether we could introduce a /dev/same device to allow
> costless huge virtual memory.
> 
> The use case is the simulation of the execution of a big irregular HPC
> application, to provision memory usage, cpu time, etc. We know how much
> time each computation loop takes, and it's easy to replace them with a
> mere accounting. We'd however like to avoid having to revamp the rest
> of the code, which does allocation/memcpys/etc., by just replacing
> the allocation calls with virtual allocations, i.e. allocations which
> return addresses of buffers that one can read/write, but the values you
> read are not necessarily what you wrote, i.e. the data is not actually
> properly stored (since we don't do the actual computations that's not a
> problem).
> 
> The way we currently do this is by some folding: we map the same normal
> file several times contiguously to form the virtual allocation. By using
> a small 1MiB file, this limits memory consumption to 1MiB plus the page
> table (and fits the dumb data in a typical cache). This however creates
> one VMA per file mapping, we get limited by the 65535 VMA limit, and
> VMA lookup becomes slow.
> 
> The way I could see is to have a /dev/same device: when you open it, it
> allocates one page. When you mmap it, it maps the same page over the
> whole resulting single VMA.
> 
> This is a quite specific use case, but it seems to be easy to implement,
> and it seems to me that it could be integrated mainline. Actually I was
> thinking that /dev/null itself could be providing that service?
> (currently it returns ENODEV)
> 
> What do people think?  Is there perhaps another solution to achieve this
> that I didn't think about?
> 
> Samuel

View attachment "dev_garbage" of type "text/plain" (5732 bytes)