linux-kernel - Re: [v4 1/1] mm: Adaptive hash table scaling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170522092910.GD8509@dhcp22.suse.cz>
Date:   Mon, 22 May 2017 11:29:10 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Pavel Tatashin <pasha.tatashin@...cle.com>
Cc:     akpm@...ux-foundation.org, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org
Subject: Re: [v4 1/1] mm: Adaptive hash table scaling

On Sat 20-05-17 13:06:53, Pavel Tatashin wrote:
[...]
>  /*
> + * Adaptive scale is meant to reduce sizes of hash tables on large memory
> + * machines. As memory size is increased the scale is also increased but at
> + * slower pace.  Starting from ADAPT_SCALE_BASE (64G), every time memory
> + * quadruples the scale is increased by one, which means the size of hash table
> + * only doubles, instead of quadrupling as well.
> + */
> +#define ADAPT_SCALE_BASE	(64ull << 30)

I have only noticed this email today because my incoming emails stopped
syncing since Friday. But this is _definitely_ not the right approachh.
64G for 32b systems is _way_ off. We have only ~1G for the kernel. I've
already proposed scaling up to 32M for 32b systems and Andi seems to be
suggesting the same. So can we fold or apply the following instead?
---
>From 6a17a022e82ac715a08a9f4707c1c29a58a2225b Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@...e.com>
Date: Mon, 22 May 2017 10:45:20 +0200
Subject: [PATCH] mm: fix adaptive hash table sizing for 32b systems

Guenter Roeck has noticed that many qemu boot test on 32b systems
and bisected it to "mm: drop HASH_ADAPT". The patch itself only
makes the HASH_ADAPT unconditional for all users which shouldn't
matter. Except it does because ADAPT_SCALE_BASE is 64GB which is
out of 32b word size so the adapt_scale loop will never terminate
and so HASH_EARLY allocations lock up with the patch while we even
do not try to use the new hash adapt code because the early allocation
suceeded.

Fix this by reducint ADAPT_SCALE_BASE down to 32MB on 32b machines.

Fixes: mm: adaptive hash table scaling
Signed-off-by: Michal Hocko <mhocko@...e.com>
---
 mm/page_alloc.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index a26e19c3e1ff..70c5fc1fb89a 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -7174,11 +7174,15 @@ static unsigned long __init arch_reserved_kernel_pages(void)
 /*
  * Adaptive scale is meant to reduce sizes of hash tables on large memory
  * machines. As memory size is increased the scale is also increased but at
- * slower pace.  Starting from ADAPT_SCALE_BASE (64G), every time memory
- * quadruples the scale is increased by one, which means the size of hash table
- * only doubles, instead of quadrupling as well.
+ * slower pace.  Starting from ADAPT_SCALE_BASE (64G on 64b systems and 32M
+ * on 32b), every time memory quadruples the scale is increased by one, which
+ * means the size of hash table only doubles, instead of quadrupling as well.
  */
+#if __BITS_PER_LONG == 64
 #define ADAPT_SCALE_BASE	(64ul << 30)
+#else
+#define ADAPT_SCALE_BASE	(32ul << 20)
+#endif
 #define ADAPT_SCALE_SHIFT	2
 #define ADAPT_SCALE_NPAGES	(ADAPT_SCALE_BASE >> PAGE_SHIFT)
 
-- 
2.11.0

-- 
Michal Hocko
SUSE Labs