linux-kernel - Re: mmap dirty limits on 32 bit kernels (Was: [BUG] New Kernel Bugs)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LFD.0.9999.0711151320010.4260@woody.linux-foundation.org>
Date:	Thu, 15 Nov 2007 13:26:13 -0800 (PST)
From:	Linus Torvalds <torvalds@...ux-foundation.org>
To:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
cc:	Bron Gondwana <brong@...tmail.fm>,
	Christian Kujau <lists@...dbynature.de>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	robm@...tmail.fm
Subject: Re: mmap dirty limits on 32 bit kernels (Was: [BUG] New Kernel
 Bugs)

On Thu, 15 Nov 2007, Linus Torvalds wrote:
> 
> The problem with HIGHMEM is that it causes various metadata (dentries, 
> inodes, page struct tables etc) to eat up memory "prime real estate" under 
> the same kind of conditions that also dirty a lot of memory. So the reason 
> we disallow HIGHMEM from dirty limits is only *partly* the per-device or 
> mapping DMA limits, and to a large degree the fact that non-highmem memory 
> is special in general, and it is usually the non-highmem areas that are 
> constrained - and need to be protected.

Final note on this (promise): 

I'd really be very interested to hear if the patch I *do* think makes 
sense (ie the removal of the old "unmapped_ratio" logic) actually already 
solves most of Bron's problems.

It may well be that that unmapped_ratio logic effectively undid the system 
configuration changes that Bron has done. It doesn't matter if Bron has

	>From our sysctl.conf:
	# This should help reduce flushing on Cache::FastMmap files
	vm.dirty_background_ratio = 50
	vm.dirty_expire_centisecs = 9000
	vm.dirty_ratio = 80
	vm.dirty_writeback_centisecs = 3000

if it turns out that the "unmapped_ratio" logic turns the 80% back down to 
5%.

It may well be that 80% of the non-highmem memory is plenty good enough! 

Sure, older kernels allowed even more of memory to be dirty (since they 
didn't count dirty mappings at all), but we may have a case where the fact 
that we discount the HIGHMEM stuff isn't the major problem in itself, and 
that the dirty_ratio sysctl should be ok - but just gets screwed over by 
that unmapped_ratio logic.

So Bron, if you can test that patch, I'd love to hear if it matters. It 
may not make any difference (maybe you don't actually trigger the 
unmapped_ratio logic at all), but I think it has the potential for being 
totally broken for you.

People that don't change the dirty_ratio from the default values would 
generally never care, because the default dirty-ratio is *already* so low 
that even if the unmapped_ratio logic triggers, it won't much matter!

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/