netdev - RE: [RFC] non-preemptible kernel socket for RAMster

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <61fd635c-d01c-46c1-ba89-2915a6ddb9f1@default>
Date:	Thu, 7 Jul 2011 08:34:09 -0700 (PDT)
From:	Dan Magenheimer <dan.magenheimer@...cle.com>
To:	"Loke, Chetan" <Chetan.Loke@...scout.com>, netdev@...r.kernel.org
Cc:	Konrad Wilk <konrad.wilk@...cle.com>, linux-mm <linux-mm@...ck.org>
Subject: RE: [RFC] non-preemptible kernel socket for RAMster

> From: Loke, Chetan [mailto:Chetan.Loke@...scout.com]
> Subject: RE: [RFC] non-preemptible kernel socket for RAMster
> 
> > -----Original Message-----
> > From: Dan Magenheimer [mailto:dan.magenheimer@...cle.com]
> >
> > > From: Loke, Chetan [mailto:Chetan.Loke@...scout.com]
> > >
> > > > From: Dan Magenheimer [mailto:dan.magenheimer@...cle.com]
> >
> > > How often are you going to re-size your remote-SWAP?
> >
> > is "as often as the working set changes on any machine in the
> > cluster", meaning *constantly*, entirely dynamically!  How
> > about a more specific example:  Suppose you have 2 machines,
> > each with 8GB of memory.  99% of the time each machine is
> > chugging along just fine and doesn't really need more than 4GB,
> > and may even use less than 1GB a large part of the time.
> > But very now and then, one of the machines randomly needs
> > 9GB, 10GB, maybe even 12GB  of memory.  This would normally
> > result in swapping.  (Most system administrators won't even
> > have this much information... they'll just know they are
> > seeing swapping and decide they need to buy more RAM.)
> >
> 
> Ok, I understand there is interest in implementing
> 'remote-volatile-ballooning-variant' but how do you pick a remote
> candidate(hypervisor)? Let's say, memory could be available on remote
> system but what if the remote-p{NIC,CPU} is overloaded? Sure, sysadmins
> won't have this info because this so dynamic(and it's quite possible as
> you mentioned above). But does the trans-remote-API know about this
> resource-availability before opening a remote-channel?
> 
> Stressing the remote-p{NIC/CPU} might trick hypervisor-vmotion-plugin to
> vmotion VM[s] to another hypervisor. How is trans-remote-API integrating
> with remote/global vmotion policies to avoid this false vmotion?

Hi Chetan --

Thanks for the continued discussion.

First, let me clarify that RAMster does not depend on virtualization.
At some time in the future, it may be a nice addition for KVM*,
but the version I am developing currently only works on a
cluster of physical machines.  So vmotion/migration is not
an issue right now


As for choosing the remote machine, another key feature of
the Transcendent Memory mechanism is that any and every page
can be rejected.  If rejected, the page remains local.  In
essence, on *every* page-to-be-swapped, machine A *asks*
machine B, "can you take this page"?  If the answer is no,
machine A can choose another machine (C), or may choose to
swap the page to its own slow swap disk.  (Currently,
only the latter is implemented, but more complicated
policy could certainly be implemented.)

Dan

* Xen doesn't have drivers so RAMster-over-network is not an option
for Xen.  A future RAMster-over-exofabric might work with Xen though.)
And, by the way, the Transcendent Memory implementation on Xen
does handle vmotion/migration so it is a solvable problem.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html