linux-kernel - RE: [PATCH -v2 -mm] add extra free kbytes tunable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <65795E11DBF1E645A09CEC7EAEE94B9CB4F747B3@USINDEVS02.corp.hds.com>
Date:	Mon, 24 Oct 2011 22:04:09 -0400
From:	Satoru Moriya <satoru.moriya@....com>
To:	David Rientjes <rientjes@...gle.com>
CC:	Rik van Riel <riel@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Randy Dunlap <rdunlap@...otime.net>,
	Satoru Moriya <smoriya@...hat.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"lwoodman@...hat.com" <lwoodman@...hat.com>,
	Seiji Aguchi <saguchi@...hat.com>,
	"hughd@...gle.com" <hughd@...gle.com>,
	"hannes@...xchg.org" <hannes@...xchg.org>
Subject: RE: [PATCH -v2 -mm] add extra free kbytes tunable

saOn 10/23/2011 05:22 PM, David Rientjes wrote:
> On Fri, 21 Oct 2011, Satoru Moriya wrote:
> 
>> We do.
>> Basically we need this kind of feature for almost all our latency
>> sensitive applications to avoid latency issue in memory allocation.
>>
> 
> These are all realtime?

Do you mean that these are all realtime process?

If so, answer is depending on the situation. In the some situations,
we can set these applications as rt-task. But the other situation,
e.g. using some middlewares, package softwares etc, we can't set them
as rt-task because they are not built for running as rt-task. And also
it is difficult to rebuilt them for working as rt-task because they
usually have huge code base.

>> Currently we run those applications on custom kernels which this
>> kind of patch is applied to. But it is hard for us to support every
>> kernel version for it. Also there are several customers who can't
>> accept a custom kernel and so they must use other commercial Unix.
>> If this feature is accepted, they will definitely use it on their
>> systems.
>>
> 
> That's precisely the problem, it's behavior is going to vary widely from 
> version to version as the implementation changes for reclaim and 
> compaction.  I think we can do much better with the priority of kswapd and 
> reclaiming above the high watermark for threads that need a surplus of 
> extra memory because they are realtime, two things we can easily do.

As I reported another mail, changing kswapd priority does not mitigate
even my simple testcase very much. Of course, reclaiming above the high
wmark may solve the issue on some workloads but if an application can
allocate memory more than high wmark - min wmark which is extended and
fast enough, latency issue will happen.
Unless this latency concern is fixed, customers doesn't use vanilla
kernel.

Regards,
Satoru--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/