linux-kernel - Re: [PATCH] add extra free kbytes tunable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20130219152936.f079c971.akpm@linux-foundation.org>
Date:	Tue, 19 Feb 2013 15:29:36 -0800
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	dormando <dormando@...ia.net>
Cc:	Rik van Riel <riel@...hat.com>,
	Seiji Aguchi <seiji.aguchi@....com>,
	Satoru Moriya <satoru.moriya@....com>,
	Randy Dunlap <rdunlap@...otime.net>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"lwoodman@...hat.com" <lwoodman@...hat.com>,
	"hughd@...gle.com" <hughd@...gle.com>
Subject: Re: [PATCH] add extra free kbytes tunable

On Sun, 17 Feb 2013 15:48:31 -0800 (PST)
dormando <dormando@...ia.net> wrote:

> Add a userspace visible knob to tell the VM to keep an extra amount
> of memory free, by increasing the gap between each zone's min and
> low watermarks.

The problem is that adding this tunable will constrain future VM
implementations.  We will forever need to at least retain the
pseudo-file.  We will also need to make some effort to retain its
behaviour.

It would of course be better to fix things so you don't need to tweak
VM internals to get acceptable behaviour.

You said:

: We have a server workload wherein machines with 100G+ of "free" memory
: (used by page cache), scattered but frequent random io reads from 12+
: SSD's, and 5gbps+ of internet traffic, will frequently hit direct reclaim
: in a few different ways.
: 
: 1) It'll run into small amounts of reclaim randomly (a few hundred
: thousand).
: 
: 2) A burst of reads or traffic can cause extra pressure, which kswapd
: occasionally responds to by freeing up 40g+ of the pagecache all at once
: (!) while pausing the system (Argh).
: 
: 3) A blip in an upstream provider or failover from a peer causes the
: kernel to allocate massive amounts of memory for retransmission
: queues/etc, potentially along with buffered IO reads and (some, but not
: often a ton) of new allocations from an application. This paired with 2)
: can cause the box to stall for 15+ seconds.

Can we prioritise these?  2) looks just awful - kswapd shouldn't just
go off and free 40G of pagecache.  Do you know what's actually in that
pagecache?  Large number of small files or small number of (very) large
files?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/