linux-kernel - Re: dentries: dentry defragmentation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100201104544.GJ29555@one.firstfloor.org>
Date:	Mon, 1 Feb 2010 11:45:44 +0100
From:	Andi Kleen <andi@...stfloor.org>
To:	Nick Piggin <npiggin@...e.de>
Cc:	Andi Kleen <andi@...stfloor.org>,
	Al Viro <viro@...IV.linux.org.uk>,
	Christoph Lameter <cl@...ux-foundation.org>,
	Dave Chinner <david@...morbit.com>,
	Alexander Viro <viro@....linux.org.uk>,
	Christoph Hellwig <hch@...radead.org>,
	Christoph Lameter <clameter@....com>,
	Rik van Riel <riel@...hat.com>,
	Pekka Enberg <penberg@...helsinki.fi>,
	akpm@...ux-foundation.org, Miklos Szeredi <miklos@...redi.hu>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Hugh Dickins <hugh@...itas.com>, linux-kernel@...r.kernel.org
Subject: Re: dentries: dentry defragmentation

On Mon, Feb 01, 2010 at 09:35:26PM +1100, Nick Piggin wrote:
> > > > > I always preferred to do defrag in the opposite way. Ie. query the
> > > > > slab allocator from existing shrinkers rather than opposite way
> > > > > around. This lets you reuse more of the locking and refcounting etc.
> > > > 
> > > > I looked at this for hwpoison soft offline.
> > > > 
> > > > But it works really badly because the LRU list ordering 
> > > > has nothing to do with the actual ordering inside the slab pages.
> > > 
> > > No, you don't *have* to follow LRU order. The most important thing
> > 
> > What list would you follow then?
> 
> You can follow the slab, as I said in the first mail.

That's pretty much what Christoph's patchkit is about (with yes some details
improved)

> 
> > There's LRU, there's hast (which is as random) and there's slab
> > itself. The only one who is guaranteed to match the physical
> > layout in memory is slab. That is what this patchkit is trying
> > to attempt.
> > 
> > > is if you followed what I wrote is to get a pin on the objects and
> > 
> > Which objects? You first need to collect all that belong to a page.
> > How else would you do that?
> 
> Objects that you're interested in reclaiming, I guess. I don't
> understand the question.

Objects that are in the same page

There are really two different cases here:
- Run out of memory: in this case i just want to find all the objects
of any page, ideally of not that recently used pages.
- I am very fragmented and want a specific page freed to get a 2MB
region back or for hwpoison:  same, but do it for a specific page.


> Right, but as you can see it is complex to do it this way. And I
> think for reclaim driven targetted reclaim, then it needn't be so
> inefficient because you aren't restricted to just one page, but
> in any page which is heavily fragmented (and by definition there
> should be a lot of them in the system).

Assuming you can identify them quickly.

> 
> Hwpoison I don't think adds much weight, frankly. Just panic and
> reboot if you get unrecoverable error. We have everything to handle

This is for soft hwpoison :- offlining pages that might go bad
in the future.

But soft hwpoison isn't the only user. The other big one would
be for large pages or other large page allocations.

-Andi
-- 
ak@...ux.intel.com -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/