linux-kernel - Re: [ofa-general] Re: Demand paging for memory regions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 13 Feb 2008 11:46:21 -0800
From:	Christian Bell <christian.bell@...gic.com>
To:	Christoph Lameter <clameter@....com>
Cc:	Jason Gunthorpe <jgunthorpe@...idianresearch.com>,
	Rik van Riel <riel@...hat.com>,
	Andrea Arcangeli <andrea@...ranet.com>, a.p.zijlstra@...llo.nl,
	izike@...ranet.com, Roland Dreier <rdreier@...co.com>,
	steiner@....com, linux-kernel@...r.kernel.org, avi@...ranet.com,
	linux-mm@...ck.org, daniel.blueman@...drics.com,
	Robin Holt <holt@....com>, general@...ts.openfabrics.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	kvm-devel@...ts.sourceforge.net
Subject: Re: [ofa-general] Re: Demand paging for memory regions

On Wed, 13 Feb 2008, Christoph Lameter wrote:

> Right. We (SGI) have done something like this for a long time with XPmem 
> and it scales ok.

I'd dispute this based on experience developing PGAS language support
on the Altix but more importantly (and less subjectively), I think
that "scales ok" refers to a very specific case.  Sure, pages (and/or
regions) can be large on some systems and the number of systems may
not always be in the thousands but you're still claiming scalability
for a mechanism that essentially logs who accesses the regions.  Then
there's the fact that reclaim becomes a collective communication
operation over all region accessors.  Makes me nervous.

> > When messages are sufficiently large, the control messaging necessary
> > to setup/teardown the regions is relatively small.  This is not
> > always the case however -- in programming models that employ smaller
> > messages, the one-sided nature of RDMA is the most attractive part of
> > it.  
> 
> The messaging would only be needed if a process comes under memory 
> pressure. As long as there is enough memory nothing like this will occur.
> 
> > Nothing any communication/runtime system can't already do today.  The
> > point of RDMA demand paging is enabling the possibility of using RDMA
> > without the implied synchronization -- the optimistic part.  Using
> > the notifiers to duplicate existing memory region handling for RDMA
> > hardware that doesn't have HW page tables is possible but undermines
> > the more important consumer of your patches in my opinion.
> 

> The notifier schemet should integrate into existing memory region 
> handling and not cause a duplication. If you already have library layers 
> that do this then it should be possible to integrate it.

I appreciate that you're trying to make a general case for the
applicability of notifiers on all types of existing RDMA hardware and
wire protocols.  Also, I'm not disagreeing whether a HW page table
is required or not: clearly it's not required to make *some* use of
the notifier scheme.

However, short of providing user-level notifications for pinned pages
that are inadvertently released to the O/S, I don't believe that the
patchset provides any significant added value for the HPC community
that can't optimistically do RDMA demand paging.

    . . christian

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/