linux-kernel - RE: [PATCH -v2 -mm] add extra free kbytes tunable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 13 Oct 2011 12:54:40 -0400
From:	Satoru Moriya <satoru.moriya@....com>
To:	David Rientjes <rientjes@...gle.com>,
	Con Kolivas <kernel@...ivas.org>
CC:	Andrew Morton <akpm@...ux-foundation.org>,
	Rik van Riel <riel@...hat.com>,
	Randy Dunlap <rdunlap@...otime.net>,
	Satoru Moriya <smoriya@...hat.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"lwoodman@...hat.com" <lwoodman@...hat.com>,
	Seiji Aguchi <saguchi@...hat.com>,
	Hugh Dickins <hughd@...gle.com>,
	"hannes@...xchg.org" <hannes@...xchg.org>
Subject: RE: [PATCH -v2 -mm] add extra free kbytes tunable

On 10/11/2011 07:22 PM, David Rientjes wrote:
> On Tue, 11 Oct 2011, Satoru Moriya wrote:
> 
> I don't know if your test case is the only thing that Rik is looking 
> at, but if so, then that statement makes me believe that this patch is 
> definitely in the wrong direction, so NACK on it until additional 
> information is presented.  The reasoning is simple: if tuning the 
> bonus given to rt-tasks in the page allocator itself would fix the 
> issue, then we can certainly add logic specifically for rt-tasks that 
> can reclaim more aggressively without needing any tunable from 
> userspace (and _certainly_ not a global tunable that affects every application!).

My test case is just a simple one (maybe too simple), and I tried
to demonstrate following issues that current kernel has with it.

1. Current kernel uses free memory as pagecache.
2. Applications may allocate memory burstly and when it happens
   they may get a latency issue because there are not enough free
   memory. Also the amount of required memory is wide-ranging.
3. Some users would like to control the amount of free memory
   to avoid the situation above.
4. User can't setup the amount of free memory explicitly.
   From user's point of view, the amount of free memory is the delta
   between high watermark - min watermark because below min watermark
   user applications incur a penalty (direct reclaim). The width of
   delta depends on min_free_kbytes, actually min watermark / 2, and
   so if we want to make free memory bigger, we must make
   min_free_kbytes bigger. It's not a intuitive and it introduces
   another problem that is possibility of direct reclaim is increased.

I think my test case is too simple and so we may be able to avoid
the latency issue with my case by setting the workload rt-task,
improving how to decide rt-task bonus and/or introducing the
Con's patches below.

But my concern described above is still alive because whether
latency issue happen or not depends on how heavily workloads
allocate memory at a short time. Of cource we can say same
things for extra_free_kbytes, but we can change it and test
an effect easily.

>>> Does there exist anything like a test case which demonstrates the 
>>> need for this feature?
>>
>> Unfortunately I don't have a real test case but just simple one.
>> And in my simple test case, I can avoid direct reclaim if we set 
>> workload as rt-task.
>>
>> The simple test case I used is following:
>> http://marc.info/?l=linux-mm&m=131605773321672&w=2
>>
> 
> I tried proposing one of Con's patches from his BFS scheduler ("mm: 
> adjust kswapd nice level for high priority page") about 1 1/2 years 
> ago that I recall and believe may significantly help your test case.  
> The thread is at http://marc.info/?t=126743860700002.  (There's a lot 
> of interesting things in Con's patchset that can be pulled into the 
> VM, this isn't the only one.)
> 
> The patch wasn't merged back then because we wanted a test case that 
> was specifically fixed by this issue, and it may be that we have just 
> found one.  If you could try it out without any extra_free_kbytes, I 
> think we may be able to help your situation.

Thank you, David. I'll try my simple workload with it.

Regards,
Satoru
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/