linux-kernel - Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1172851477.9640.26.camel@localhost.localdomain>
Date:	Fri, 02 Mar 2007 17:04:36 +0100
From:	Brice Figureau <brice+lklm@...sofwonder.com>
To:	Leroy van Logchem <leroy.vanlogchem@...elft.nl>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: [RFC][PATCH 0/3] VM throttling: avoid blocking
	occasional	writers

Hi,

On Fri, 2007-03-02 at 13:06 +0000, Leroy van Logchem wrote:
> > I'm sorry to piggy-back this thread.
> > 
> > Could it be what I'm experiencing in the following bugzilla report:
> > http://bugzilla.kernel.org/show_bug.cgi?id=7372
> > 
> > As I explained in the report, I see this issue only since 2.6.18.
> > So if your concern is related to mine, what could have changed between
> > 2.6.17 and 2.6.18 related to this?
> 
> I don't think it's 2.6.x related, it's been under the sheets from start.

Maybe. Still the issue has been aggravated between 2.6.17 and 2.6.18.
Right now (running 2.6.17.13) I can backup and do whatever I want, even
with high memory pressure because of mysql (which is consumming right
now about 3.8GB of 4GB).

Under 2.6.18 and later it is simply impossible to perform that (except
with the dd directIO trick).
This makes me think that something else has joined the party on
2.6.18...

> Related to your problem in the 7372 bug:
> 
> Pages are kept in memory for re-use, which is fast and fine except for:
> 1) data without re-use value or even single use
> 2) applications _do not_ advise the kernel how to cache pages related
>    to there self generated i/o. POSIX does provide mechanisms to
>    properly do so. But the kernel should help these poor apps.
>
> To minimize your MySQL backup cp(1) problem, try this workaround:
> 
> cat ./cp_direct.sh 
> #!/bin/sh
> dd if=$1 of=$2 bs=1M iflag=direct oflag=direct

Yes, I already did this, and it helped, see:
http://bugzilla.kernel.org/show_bug.cgi?id=7372#c16

> Combine this with [/etc/sysctl.conf]:
> 
> vm.vfs_cache_pressure = 1
> vm.dirty_ratio = 2
> vm.dirty_background_ratio = 1
> 
> This should reduce both the stress on the vm and response latency during
> interactive work.

I'll test the vfs_cache_pressure (I was already using those dirty_*
settings) whenever I'll reboot the server to 2.6.20.

Please CC: me on list replies as I'm not subscribed to the list.

Thanks for the tips.
Regards
-- 
Brice Figureau <brice+lklm@...sofwonder.com>

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/