linux-kernel - Re: [PATCH 4/8] Shrink zcache based on memlimit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201007210732.03909.edt@aei.ca>
Date:	Wed, 21 Jul 2010 07:32:02 -0400
From:	Ed Tomlinson <edt@....ca>
To:	ngupta@...are.org
Cc:	Minchan Kim <minchan.kim@...il.com>,
	Pekka Enberg <penberg@...helsinki.fi>,
	Hugh Dickins <hugh.dickins@...cali.co.uk>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Greg KH <greg@...ah.com>,
	Dan Magenheimer <dan.magenheimer@...cle.com>,
	Rik van Riel <riel@...hat.com>, Avi Kivity <avi@...hat.com>,
	Christoph Hellwig <hch@...radead.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
	"linux-mm" <linux-mm@...ck.org>,
	"linux-kernel" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 4/8] Shrink zcache based on memlimit

On Wednesday 21 July 2010 00:52:40 Nitin Gupta wrote:
> On 07/21/2010 04:33 AM, Minchan Kim wrote:
> > On Fri, Jul 16, 2010 at 9:37 PM, Nitin Gupta <ngupta@...are.org> wrote:
> >> User can change (per-pool) memlimit using sysfs node:
> >> /sys/kernel/mm/zcache/pool<id>/memlimit
> >>
> >> When memlimit is set to a value smaller than current
> >> number of pages allocated for that pool, excess pages
> >> are now freed immediately instead of waiting for get/
> >> flush for these pages.
> >>
> >> Currently, victim page selection is essentially random.
> >> Automatic cache resizing and better page replacement
> >> policies will be implemented later.
> > 
> > Okay. I know this isn't end. I just want to give a concern before you end up.
> > I don't know how you implement reclaim policy.
> > In current implementation, you use memlimit for determining when reclaim happen.
> > But i think we also should follow global reclaim policy of VM.
> > I means although memlimit doen't meet, we should reclaim zcache if
> > system has a trouble to reclaim memory.
> 
> Yes, we should have a way to do reclaim depending on system memory pressure
> and also when user explicitly wants so i.e. when memlimit is lowered manually.
> 
> > AFAIK, cleancache doesn't give any hint for that. so we should
> > implement it in zcache itself.
> 
> I think cleancache should be kept minimal so yes, all reclaim policies should
> go in zcache layer only.
> 
> > At first glance, we can use shrink_slab or oom_notifier. But both
> > doesn't give any information of zone although global reclaim do it by
> > per-zone.
> > AFAIK, Nick try to implement zone-aware shrink slab. Also if we need
> > it, we can change oom_notifier with zone-aware oom_notifier. Now it
> > seems anyone doesn't use oom_notifier so I am not sure it's useful.
> > 
> 
> I don't think we need these notifiers as we can simply create a thread
> to monitor cache hit rate, system memory pressure etc. and shrink/expand
> the cache accordingly.

Nitin,

Based on experience gained when adding the shrinker callbacks, I would 
strongly recommend you use them.  I tried several hacks along the lines of
what you are proposing before moving settling on the callbacks.  They 
are effective and make sure that memory is released when its required.
What would happen with the other methods is that memory would either 
not be released or would be released when it was not needed.

Thanks
Ed Tomlinson.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/