linux-kernel - Re: [PATCH 06/14] Export fragmentation index via /proc/extfrag

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100407104632.GP17882@csn.ul.ie>
Date:	Wed, 7 Apr 2010 11:46:32 +0100
From:	Mel Gorman <mel@....ul.ie>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Andrea Arcangeli <aarcange@...hat.com>,
	Christoph Lameter <cl@...ux-foundation.org>,
	Adam Litke <agl@...ibm.com>, Avi Kivity <avi@...hat.com>,
	David Rientjes <rientjes@...gle.com>,
	Minchan Kim <minchan.kim@...il.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Rik van Riel <riel@...hat.com>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [PATCH 06/14] Export fragmentation index via
	/proc/extfrag_index

On Tue, Apr 06, 2010 at 05:05:42PM -0700, Andrew Morton wrote:
> On Fri,  2 Apr 2010 17:02:40 +0100
> Mel Gorman <mel@....ul.ie> wrote:
> 
> > Fragmentation index is a value that makes sense when an allocation of a
> > given size would fail. The index indicates whether an allocation failure is
> > due to a lack of memory (values towards 0) or due to external fragmentation
> > (value towards 1).  For the most part, the huge page size will be the size
> > of interest but not necessarily so it is exported on a per-order and per-zone
> > basis via /proc/extfrag_index
> 
> (/proc/sys/vm?)
> 

It can move.

> Like unusable_index, this seems awfully specialised. 

Except in this case, the fragmentation index is used by the kernel when
deciding in advance whether compaction will do the job or if lumpy
reclaim is required.

I could avoid exposing this to userspace but it would make it harder to
decide what needs to happen with extfrag_threshold later. i.e. does the
threshold need a different value (proc would help gather the data) or
is a new heuristic needed.

> Perhaps we could
> hide it under CONFIG_MEL, or even put it in debugfs with the intention
> of removing it in 6 or 12 months time.  Either way, it's hard to
> justify permanently adding this stuff to every kernel in the world?
> 

Moving it to debugfs would satisfy the requirement of tuning extfrag_threshold
without adding it to every kernel but it could also be just removed.

> 
> I have a suspicion that all the info in unusable_index and
> extfrag_index could be computed from userspace using /proc/kpageflags

It can be computed from buddyinfo. I used a perl script to calculate it
in the past. I exposed the information from in-kernel in these patches so
people would be guaranteed to have the same information as me.

> (and perhaps a bit of dmesg-diddling to find the zones). 

Can be figured out from buddyinfo too.

> If that can't
> be done today, I bet it'd be pretty easy to arrange for it.
> 

It is. Will I just remove the proc files, keep the internal calculation
for fragmentation_index and kick that perl script into shape to produce
the same information from buddyinfo?

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/