linux-kernel - Re: [PATCH - RFC] allow setting vm_dirty below 1% for large memory machines

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20070111122127.5bcc0b0f.akpm@osdl.org>
Date:	Thu, 11 Jan 2007 12:21:27 -0800
From:	Andrew Morton <akpm@...l.org>
To:	dean gaudet <dean@...tic.org>
Cc:	Neil Brown <neilb@...e.de>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH - RFC] allow setting vm_dirty below 1% for large memory
 machines

On Thu, 11 Jan 2007 03:04:00 -0800 (PST)
dean gaudet <dean@...tic.org> wrote:

> On Tue, 9 Jan 2007, Neil Brown wrote:
> 
> > Imagine a machine with lots of memory - say 100Gig.
> 
> i've had these problems on machines as "small" as 8GiB.  the real problem 
> is that the kernel will let millions of potential (write) IO ops stack up 
> for a device which can handle only mere 100s of IOs per second.  (and i'm 
> not convinced it does the IOs in a sane order when it has millions to 
> choose from)
> 
> replacing the percentage based dirty_ratio / dirty_background_ratio with 
> sane kibibyte units is a good fix... but i'm not sure it's sufficient.
> 
> it seems like the "flow control" mechanism (i.e. dirty_ratio) should be on 
> a device basis...
> 
> try running doug ledford'd memtest.sh on an 8GiB box with a single disk, 
> let it go a few minutes then ^C and type "sync".  i've had to wait 10 
> minutes (2.6.18 with default vm settings).
> 
> it makes it hard to guarantee a box can shutdown quickly -- nasty for 
> setting up UPS on-battery timeouts for example.
> 

Increasing the request queue size should help there
(/sys/block/sda/queue/nr_requests).  Maybe 25% or more benefit with that
test, at a guess.

Probably initscripts should do that rather than leaving the kernel defaults
in place.  It's a bit tricky for the kernel to do because the decision
depends upon the number of disks in the system, as well as the amount of
memory.

Or perhaps the kernel should implement a system-wide limit on the number of
requests in flight.  While avoiding per-device starvation.  Tricky.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/