linux-kernel - RE: [GIT PULL] mm: frontswap (for 3.2 window)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1320221686.3091.40.camel@dabdike>
Date:	Wed, 02 Nov 2011 12:14:46 +0400
From:	James Bottomley <James.Bottomley@...senPartnership.com>
To:	Dan Magenheimer <dan.magenheimer@...cle.com>
Cc:	Andrea Arcangeli <aarcange@...hat.com>,
	Pekka Enberg <penberg@...nel.org>,
	Cyclonus J <cyclonusj@...il.com>,
	Sasha Levin <levinsasha928@...il.com>,
	Christoph Hellwig <hch@...radead.org>,
	David Rientjes <rientjes@...gle.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	linux-mm@...ck.org, LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Konrad Wilk <konrad.wilk@...cle.com>,
	Jeremy Fitzhardinge <jeremy@...p.org>,
	Seth Jennings <sjenning@...ux.vnet.ibm.com>, ngupta@...are.org,
	Chris Mason <chris.mason@...cle.com>, JBeulich@...ell.com,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Jonathan Corbet <corbet@....net>
Subject: RE: [GIT PULL] mm: frontswap (for 3.2 window)

On Tue, 2011-11-01 at 11:21 -0700, Dan Magenheimer wrote:
> > From: James Bottomley [mailto:James.Bottomley@...senPartnership.com]
> > Subject: Re: [GIT PULL] mm: frontswap (for 3.2 window)
> > 
> > Actually, I think there's an unexpressed fifth requirement:
> > 
> > 5. The optimised use case should be for non-paging situations.
> 
> Not quite sure what you mean here (especially for frontswap)...

I mean could it be used in a more controlled situation than an
alternative to swap?

> > The problem here is that almost every data centre person tries very hard
> > to make sure their systems never tip into the swap zone.  A lot of
> > hosting datacentres use tons of cgroup controllers for this and
> > deliberately never configure swap which makes transcendent memory
> > useless to them under the current API.  I'm not sure this is fixable,
> 
> I can't speak for cgroups, but the generic "state-of-the-art"
> that you describe is a big part of what frontswap DOES try
> to fix, or at least ameliorate.  Tipping "into the swap zone"
> is currently very bad.  Very very bad.  Frontswap doesn't
> "solve" swapping, but it is the foundation for some of the
> first things in a long time that aren't just "add more RAM."

OK, I still don't think you understand what I'm saying.  Machines in a
Data Centre tend to be provisioned to criticality.  What this means is
that the Data Centre has a bunch of mandatory work and a bunch of Best
Effort work (and grades in between).  We load up the mandatory work
according to the resource limits being careful not to overprovision the
capacity then we look at the spare capacity and slot in the Best effort
stuff.  We want the machine to run at capacity, not over it; plus we
need to respond instantly for demands of the mandatory work, which
usually involves either dialling down or pushing away best effort work.
In this situation, action is taken long before the swap paths become
active because if they activate, the entire machine bogs and you've just
blown the SLA on the mandatory work.

This is why a lot of data centres simply never configure swap for this
reason.  Putting frontswap in the swap paths means that the data centre
job scheduler has taken action long before frontswap ever activates, so
it can never be used which is why I wrote the above.  

> > but it's the reason why a large swathe of users would never be
> > interested in the patches, because they by design never operate in the
> > region transcended memory is currently looking to address.
> 
> It's true, those that are memory-rich and can spend nearly
> infinite amounts on more RAM (and on high-end platforms that
> can expand to hold massive amounts of RAM) are not tmem's
> target audience.

Where do you get the infinite RAM idea from?  The most concrete example
of what I said above are Lean Data Centres, which are highly resource
constrained but they want to run at (or just below) criticality so that
they get through all of the Mandatory and as much of the best effort
work as they can.

> > This isn't an inherent design flaw, but it does ask the question "is
> > your design scope too narrow?"
> 
> Considering all the hazing that I've gone through to get
> this far, you think I should _expand_ my design scope?!? :-)
> Thanks, I guess I'll pass. :-)

Sure, I think the conclusion that Transcendent Memory has no
applicability to a lean Data Centre isn't unreasonable; I was just
probing to see if it was the only conclusion.

James

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/