linux-kernel - Re: [PATCH 06/11] Export fragmentation index via /proc/extfrag

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20100326120844.6C9B.A69D9226@jp.fujitsu.com>
Date:	Fri, 26 Mar 2010 12:10:34 +0900 (JST)
From:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
To:	Mel Gorman <mel@....ul.ie>
Cc:	kosaki.motohiro@...fujitsu.com,
	Andrew Morton <akpm@...ux-foundation.org>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Christoph Lameter <cl@...ux-foundation.org>,
	Adam Litke <agl@...ibm.com>, Avi Kivity <avi@...hat.com>,
	David Rientjes <rientjes@...gle.com>,
	Rik van Riel <riel@...hat.com>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [PATCH 06/11] Export fragmentation index via /proc/extfrag_index

> On Thu, Mar 25, 2010 at 08:20:04PM +0900, KOSAKI Motohiro wrote:
> > > On Thu, Mar 25, 2010 at 11:47:17AM +0900, KOSAKI Motohiro wrote:
> > > > > On Tue, Mar 23, 2010 at 09:22:04AM +0900, KOSAKI Motohiro wrote:
> > > > > > > > > +	/*
> > > > > > > > > +	 * Index is between 0 and 1 so return within 3 decimal places
> > > > > > > > > +	 *
> > > > > > > > > +	 * 0 => allocation would fail due to lack of memory
> > > > > > > > > +	 * 1 => allocation would fail due to fragmentation
> > > > > > > > > +	 */
> > > > > > > > > +	return 1000 - ( (1000+(info->free_pages * 1000 / requested)) / info->free_blocks_total);
> > > > > > > > > +}
> > > > > > > > 
> > > > > > > > Dumb question.
> > > > > > > > your paper (http://portal.acm.org/citation.cfm?id=1375634.1375641) says
> > > > > > > > fragmentation_index = 1 - (TotalFree/SizeRequested)/BlocksFree
> > > > > > > > but your code have extra '1000+'. Why?
> > > > > > > 
> > > > > > > To get an approximation to three decimal places.
> > > > > > 
> > > > > > Do you mean this is poor man's round up logic?
> > > > > 
> > > > > Not exactly.
> > > > > 
> > > > > The intention is to have a value of 968 instead of 0.968231. i.e.
> > > > > instead of a value between 0 and 1, it'll be a value between 0 and 1000
> > > > > that matches the first three digits after the decimal place.
> > > > 
> > > > Let's consider extream case.
> > > > 
> > > > free_pages: 1
> > > > requested: 1
> > > > free_blocks_total: 1
> > > > 
> > > > frag_index = 1000  - ((1000 + 1*1000/1))/1 = -1000
> > > > 
> > > > This is not your intension, I guess. 
> > > 
> > > Why not?
> > > 
> > > See this comment
> > > 
> > > /* Fragmentation index only makes sense when a request would fail */
> > > 
> > > In your example, there is a free page of the requested size so the allocation
> > > would succeed. In this case, fragmentation index does indeed go negative
> > > but the value is not useful.
> > >
> > > > Probably we don't need any round_up/round_down logic. because fragmentation_index
> > > > is only used "if (fragindex >= 0 && fragindex <= 500)" check in try_to_compact_pages().
> > > > +1 or -1 inaccurate can be ignored. iow, I think we can remove '1000+' expression.
> > > > 
> > > 
> > > This isn't about rounding, it's about having a value that normally is
> > > between 0 and 1 expressed as a number between 0 and 1000 because we
> > > can't use double in the kernel.
> > 
> > Sorry, My example was wrong. new example is here.
> > 
> > free_pages: 4
> > requested: 2
> > free_blocks_total: 4
> > 
> > theory: 1 - (TotalFree/SizeRequested)/BlocksFree
> >             = 1 - (4/2)/4 = 0.5
> > 
> > code : 1000 - ((1000 + 4*1000/2))/4 = 1000 - (1000 + 2000)/4 = 1000/4 = 250
> > 
> > I don't think this is three decimal picking up code. This seems might makes
> > lots compaction invocation rather than theory.
> > 
> 
> Ok, I cannot apologise for this enough.
> 
> Since that paper was published, further work showed that the equation could
> be much improved. As part of that, I updated the equation to the following;
> 
> double index = 1    - ( (1    + ((double)info->free_pages        / requested)) / info->free_blocks_total);
> 
> or when approximated to three decimal places
> 
> int index =    1000 - ( (1000 + (        info->free_pages * 1000 / requested)) / info->free_blocks_total);
> 
> Your analysis of the paper is perfect. When slotted into a driver program
> with your example figures, I get the following results
> 
> old equation = 0.500000
> current equation = 0.250000
> integer approximation = 250
> 
> The code as-is is correct and is what I intended. My explanation on the
> other hand sucks and I should have remembered that I updated equation since
> I published that paper 2 years ago.
> 
> Again, I am extremely sorry for misleading you.

No worry at all. it is merely review. I have no objection this equation if it is intentional. :)



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/