linux-kernel - 13GB dcache+inode cache hash tables

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 25 Jun 2013 16:56:26 +0800
From:	Daniel J Blueman <daniel@...ascale-asia.com>
To:	Linux Kernel <linux-kernel@...r.kernel.org>
CC:	Ingo Molnar <mingo@...hat.com>, Steffen Persvold <sp@...ascale.com>
Subject: 13GB dcache+inode cache hash tables

As memory capacity increases, we see the dentry and inode cache hash 
tables grow to wild sizes [1], eg 13GB is consumed on a 4.5TB system.

Perhaps a better approach adds a linear component to an exponent to give 
tuned scaling, given that spatial locality is an advantage in hash table 
and careful use of resources.

The same approach would fit to other hash tables (mount-cache, TCP 
established, TCP bind, UDP, UDP-Lite, Dquot-cache) with different 
coefficients, so perhaps we could generalise.

If so what are reasonable reference points and assumptions?

Thanks,
   Daniel

--- [1]

1GB:
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)

8GB:
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)

1TB:
Dentry cache hash table entries: 134217728 (order: 18, 1073741824 bytes)
Inode-cache hash table entries: 67108864 (order: 17, 536870912 bytes)

4.5TB
Dentry cache hash table entries: 1073741824 (order: 21, 8589934592 bytes)
Inode-cache hash table entries: 536870912 (order: 20, 4294967296 bytes)
-- 
Daniel J Blueman
Principal Software Engineer, Numascale Asia
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/