linux-kernel - Re: [RFC][PATCH 0/2] Tunable watermark

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.00.1101071416450.23577@chino.kir.corp.google.com>
Date:	Fri, 7 Jan 2011 14:23:59 -0800 (PST)
From:	David Rientjes <rientjes@...gle.com>
To:	Satoru Moriya <satoru.moriya@....com>
cc:	linux-mm@...ck.org, linux-kernel@...r.kernel.org,
	linux-doc@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Mel Gorman <mel@....ul.ie>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Randy Dunlap <rdunlap@...otime.net>,
	dle-develop@...ts.sourceforge.net,
	Seiji Aguchi <seiji.aguchi@....com>
Subject: Re: [RFC][PATCH 0/2] Tunable watermark

On Fri, 7 Jan 2011, Satoru Moriya wrote:

> This patchset introduces a new knob to control each watermark
> separately.
> 
> [Purpose]
> To control the timing at which kswapd/direct reclaim starts(ends)
> based on memory pressure and/or application characteristics
> because direct reclaim makes a memory alloc/access latency worse.
> (We'd like to avoid direct reclaim to keep latency low even if
>  under the high memory pressure.)
> 
> [Problem]
> The thresholds kswapd/direct reclaim starts(ends) depend on
> watermark[min,low,high] and currently all watermarks are set
> based on min_free_kbytes. min_free_kbytes is the amount of
> free memory that Linux VM should keep at least.
> 

Not completely, it also depends on the amount of lowmem (because of the 
reserve setup next) and the amount of memory in each zone.

> This means the difference between thresholds at which kswapd
> starts and direct reclaim starts depends on the amount of free
> memory.
> 
> On the other hand, the amount of required memory depends on
> applications. Therefore when it allocates/access memory more
> than the difference between watemark[low] and watermark[min],
> kernel sometimes runs direct reclaim before allocation and
> it makes application latency bigger.
> 
> [Solution]
> To avoid the situation above, this patch set introduces new
> tunables /proc/sys/vm/wmark_min_kbytes, wmark_low_kbytes and
> wmark_high_kbytes. Each entry controls watermark[min],
> watermark[low] and watermark[high] separately.
> By using these parameters one can make the difference between
> min and low bigger than the amount of memory which applications
> require.
> 

I really dislike this because it adds additional tunables that should 
already be handled correctly by the VM and it's very difficult for users 
to know what to tune these values to; these watermarks (with the exception 
of min) are supposed to be internal to the VM implementation.

You didn't mention why it wouldn't be possible to modify 
setup_per_zone_wmarks() in some way for your configuration so this happens 
automatically.  If you can find a deterministic way to set these 
watermarks from userspace, you should be able to do it in the kernel as 
well based on the configuration.

I think we should invest time in making sure the VM works for any type of 
workload thrown at it instead of relying on userspace making lots of 
adjustments.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/