linux-kernel - Re: [RFC][PATCH] page reclaim throttle take2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 27 Feb 2008 13:29:18 +0530
From:	Balbir Singh <balbir@...ux.vnet.ibm.com>
To:	David Rientjes <rientjes@...gle.com>
CC:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	Rik van Riel <riel@...hat.com>,
	Lee Schermerhorn <Lee.Schermerhorn@...com>,
	Nick Piggin <npiggin@...e.de>
Subject: Re: [RFC][PATCH] page reclaim throttle take2

David Rientjes wrote:
> On Wed, 27 Feb 2008, Balbir Singh wrote:
> 
>> Since we're talking of parallel reclaims, I think it's a function of CPUs and
>> Nodes. I'd rather keep it as a sysctl with a good default value based on the
>> topology. If we end up getting it wrong, the system administrator has a choice.
>> That is better than expecting him/her to recompile the kernel and boot that. A
>> sysctl does not create problems either w.r.t changing the number of threads, no
>> hard to solve race-conditions - it is fairly straight forward
>>
> 
> We lack node hotplug, so the dependence on the number of system nodes in 
> the equation is static and can easily be defined at compile-time.
> 

Let's forget node hotplug for the moment, but what if someone

1. Changes the machine configuration and adds more nodes, do we expect the
kernel to be recompiled? Or is it easier to update /etc/sysctl.conf?
2. Uses fake NUMA nodes and increases/decreases the number of nodes across
reboots. Should the kernel be recompiled?

> I agree that the maximum number of parallel reclaim threads should be a 
> function of cpus, so you can easily make it that by adding callback 
> functions for cpu hotplug events.
> 
> Perhaps a better alternative than creating a set of heuristics and setting 
> a user-defined maximum on the number of concurrent reclaim threads is to 
> configure the number of threads to be used for each online cpu called 
> CONFIG_NUM_RECLAIM_THREADS_PER_CPU.  This solves the lock contention 
> problem if configured properly that was mentioned earlier.
> 

I am afraid it doesn't. Consider as you scale number of CPU's with the same
amount of memory, we'll end up making the reclaim problem worse.

> Adding yet another sysctl for this functionality seems unnecessary, unless 
> it is attempting to address other VM problems where page reclaim needs to 
> be throttled when it is being stressed.  Those issues need to be addressed 
> directly, in my opinion, instead of attempting to workaround it by 
> limiting the number of concurrent reclaim threads.

We are providing a solution with a good default value, allowing the
administrator to change them when our defaults don't work well.

-- 
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/