lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Thu, 29 Sep 2011 14:40:15 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	Michel Lespinasse <walken@...gle.com>
Cc:	linux-mm@...ck.org, linux-kernel@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Rik van Riel <riel@...hat.com>,
	Balbir Singh <bsingharora@...il.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Johannes Weiner <jweiner@...hat.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Hugh Dickins <hughd@...gle.com>,
	Michael Wolf <mjwolf@...ibm.com>
Subject: Re: [PATCH 2/9] kstaled: documentation and config option.

On Wed, 28 Sep 2011 16:48:44 -0700
Michel Lespinasse <walken@...gle.com> wrote:

> On Tue, Sep 27, 2011 at 11:53 PM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@...fujitsu.com> wrote:
> > On Tue, 27 Sep 2011 17:49:00 -0700
> > Michel Lespinasse <walken@...gle.com> wrote:
> >> +* idle_2_clean, idle_2_dirty_file, idle_2_dirty_swap: same definitions as
> >> +  above, but for pages that have been untouched for at least two scan cycles.
> >> +* these fields repeat up to idle_240_clean, idle_240_dirty_file and
> >> +  idle_240_dirty_swap, allowing one to observe idle pages over a variety
> >> +  of idle interval lengths. Note that the accounting is cumulative:
> >> +  pages counted as idle for a given interval length are also counted
> >> +  as idle for smaller interval lengths.
> >
> > I'm sorry if you've answered already.
> >
> > Why 240 ? and above means we have idle_xxx_clean/dirty/ xxx is 'seq 2 240' ?
> > Isn't it messy ? Anyway, idle_1_clean etc should be provided.
> 
> We don't have all values - we export values for 1, 2, 5, 15, 30, 60,
> 120 and 240 idle scan intervals.
> In our production setup, the scan interval is set at 120 seconds.
> The exported histogram values are chosen so that each is approximately
> double as the previous, and they align with human units i.e. 30 scan
> intervals == 1 hour.
> We use one byte per page to track the number of idle cycles, which is
> why we don't export anything over 255 scan intervals
> 

If LRU is divided into 1,2,5,15,30,60,120,240 intervals, ok, I think having
this statistics in the kernel means something..
Do you have any plan to using the aging value for global LRU scheduling ?


BTW, how about having 'aging' and 'histgram' on demand ?

Now, you do all scan by a thread and does aging by counter. But having
   - scan thread per interval
   - alloc bitmap (for PG_young, PG_idle) per scan thread.
will allow you to have arbitrary scan_interval/histgram and to avoid
to have unnecessary data.
 
Then, the users can get the histgram they want. Users will be able to
get 12h, 24h histgram. But each threads will use 2bit per pages.

Off topic:
you allocated 'aging' array in pgdat. please allocate it per secion
if CONFIG_SPARSEMEM. Then, you can handle memory hotplug easily.


Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists