linux-kernel - Re: Disabling in-memory write cache for x86-64 in Linux II

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20131025230545.GB31280@localhost>
Date:	Sat, 26 Oct 2013 00:05:45 +0100
From:	Fengguang Wu <fengguang.wu@...el.com>
To:	Theodore Ts'o <tytso@....edu>,
	"Artem S. Tashkinov" <t.artem@...os.com>,
	torvalds@...ux-foundation.org, akpm@...ux-foundation.org,
	linux-kernel@...r.kernel.org
Cc:	Diego Calleja <diegocg@...il.com>, David Lang <david@...g.hm>,
	NeilBrown <neilb@...e.de>
Subject: Re: Disabling in-memory write cache for x86-64 in Linux II

On Fri, Oct 25, 2013 at 05:18:42AM -0400, Theodore Ts'o wrote:
> On Fri, Oct 25, 2013 at 08:30:53AM +0000, Artem S. Tashkinov wrote:
> > My feeling is that vm.dirty_ratio/vm.dirty_background_ratio should _not_ be
> > percentage based, 'cause for PCs/servers with a lot of memory (say 64GB or
> > more) this value becomes unrealistic (13GB) and I've already had some
> > unpleasant effects due to it.
> 
> What I think would make sense is to dynamically measure the speed of
> writeback, so that we can set these limits as a function of the device
> speed.  It's already the case that the writeback limits don't make
> sense on a slow USB 2.0 storage stick; I suspect that for really huge
> RAID arrays or very fast flash devices, it doesn't make much sense
> either.
> 
> The problem is that if you have a system that has *both* a USB stick
> _and_ a fast flash/RAID storage array both needing writeback, this
> doesn't work well --- but what we have right now doesn't work all that
> well anyway.

Ted, when trying to follow up your email, I got a crazy idea and it'd
be better throw it out rather than carrying it to bed. :)

We could do per-bdi dirty thresholds - which has been proposed 1-2
times before by different people.

The per-bdi dirty thresholds could be auto set by the kernel this way: 
start it with an initial value of 100MB. When reached, put all the
100MB dirty data to IO and get an estimation of the write bandwidth.
>From then on, set the bdi's dirty threshold to N * bdi_write_bandwidth,
where N is the seconds of dirty data we'd like to cache in memory.

Thanks,
Fengguang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/