linux-kernel - Re: [RFC PATCH 3/3] mm/map_contig: Add mmap(MAP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 16 Oct 2017 12:53:41 -0500 (CDT)
From:   Christopher Lameter <cl@...ux.com>
To:     Michal Hocko <mhocko@...nel.org>
cc:     Pavel Machek <pavel@....cz>,
        Mike Kravetz <mike.kravetz@...cle.com>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, linux-api@...r.kernel.org,
        Marek Szyprowski <m.szyprowski@...sung.com>,
        Michal Nazarewicz <mina86@...a86.com>,
        "Aneesh Kumar K . V" <aneesh.kumar@...ux.vnet.ibm.com>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Guy Shattah <sguy@...lanox.com>,
        Anshuman Khandual <khandual@...ux.vnet.ibm.com>,
        Laura Abbott <labbott@...hat.com>,
        Vlastimil Babka <vbabka@...e.cz>
Subject: Re: [RFC PATCH 3/3] mm/map_contig: Add mmap(MAP_CONTIG) support

On Mon, 16 Oct 2017, Michal Hocko wrote:

> On Mon 16-10-17 11:02:24, Cristopher Lameter wrote:
> > On Mon, 16 Oct 2017, Michal Hocko wrote:
> >
> > > > So I mmap(MAP_CONTIG) 1GB working of working memory, prefer some data
> > > > structures there, maybe recieve from network, then decide to write
> > > > some and not write some other.
> > >
> > > Why would you want this?
> >
> > Because we are receiving a 1GB block of data and then wan to write it to
> > disk. Maybe we want to modify things a bit and may not write all that we
> > received.
>
> And why do you need that in a single contiguous numbers? If performance,
> do you have any numbers that would clearly tell the difference?

Again we have that in the presentation. Why keep asking the same question
if you already have the answer multiple times?

1G of data requires 250000 page structs to handle if the memory is not
contiguous. This is more than most controllers can support and thus the
overhead will dominate I/O. Also the scatter gather lists will cover lots
of linked 4k pages even to manage.

And in practice we already have multiple gigabytes per requests which
makes it even more severe. You cannot do a "cp" operation anymore. Instead
you need to have special code that allocates huge pages, does direct I/O
etc etc,