| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20090415105033.AC29.A69D9226@jp.fujitsu.com>
Date: Wed, 15 Apr 2009 11:04:59 +0900 (JST)
From: KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
To: Dave Hansen <dave@...ux.vnet.ibm.com>
Cc: kosaki.motohiro@...fujitsu.com, linux-mm <linux-mm@...ck.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
Eric B Munson <ebmunson@...ibm.com>,
Mel Gorman <mel@...ux.vnet.ibm.com>,
Christoph Lameter <cl@...ux-foundation.org>
Subject: Re: meminfo Committed_AS underflows
> I have a set of ppc64 machines that seem to spontaneously get underflows
> in /proc/meminfo's Committed_AS field:
>
> # while true; do cat /proc/meminfo | grep _AS; sleep 1; done | uniq -c
> 1 Committed_AS: 18446744073709323392 kB
> 11 Committed_AS: 18446744073709455488 kB
> 6 Committed_AS: 35136 kB
> 5 Committed_AS: 18446744073709454400 kB
> 7 Committed_AS: 35904 kB
> 3 Committed_AS: 18446744073709453248 kB
> 2 Committed_AS: 34752 kB
> 9 Committed_AS: 18446744073709453248 kB
> 8 Committed_AS: 34752 kB
> 3 Committed_AS: 18446744073709320960 kB
> 7 Committed_AS: 18446744073709454080 kB
> 3 Committed_AS: 18446744073709320960 kB
> 5 Committed_AS: 18446744073709454080 kB
> 6 Committed_AS: 18446744073709320960 kB
>
> As you can see, it bounces in and out of it. I think the problem is
> here:
>
> #define ACCT_THRESHOLD max(16, NR_CPUS * 2)
> ...
> void vm_acct_memory(long pages)
> {
> long *local;
>
> preempt_disable();
> local = &__get_cpu_var(committed_space);
> *local += pages;
> if (*local > ACCT_THRESHOLD || *local < -ACCT_THRESHOLD) {
> atomic_long_add(*local, &vm_committed_space);
> *local = 0;
> }
> preempt_enable();
> }
>
> Plus, some joker set CONFIG_NR_CPUS=1024.
>
> nr_cpus (1024) * 2 * page_size (64k) = 128MB. That means each cpu can
> skew the counter by 128MB. With 1024 CPUs that means that we can have
> ~128GB of outstanding percpu accounting that meminfo doesn't see. Let's
> say we do vm_acct_memory(128MB-1) on 1023 of the CPUs, then on the other
> CPU, we do vm_acct_memory(-128GB).
>
> The 1023 cpus won't ever hit the ACCT_THRESHOLD. The 1 CPU that did
> will decrement the global 'vm_committed_space' by ~128 GB. Underflow.
> Yay. This happens on a much smaller scale now.
>
> Should we be protecting meminfo so that it spits slightly more sane
> numbers out to the user?
Can you try to this patch? (Oh well, I can't reproduce this underflow
on my small machine)
===============
Dave Hansen reported committed_AS field can underfolow.
> # while true; do cat /proc/meminfo | grep _AS; sleep 1; done | uniq -c
> 1 Committed_AS: 18446744073709323392 kB
> 11 Committed_AS: 18446744073709455488 kB
> 6 Committed_AS: 35136 kB
> 5 Committed_AS: 18446744073709454400 kB
> 7 Committed_AS: 35904 kB
> 3 Committed_AS: 18446744073709453248 kB
> 2 Committed_AS: 34752 kB
> 9 Committed_AS: 18446744073709453248 kB
> 8 Committed_AS: 34752 kB
> 3 Committed_AS: 18446744073709320960 kB
> 7 Committed_AS: 18446744073709454080 kB
> 3 Committed_AS: 18446744073709320960 kB
> 5 Committed_AS: 18446744073709454080 kB
> 6 Committed_AS: 18446744073709320960 kB
Because NR_CPU can be greater than 1000. and meminfo_proc_show()
doesn't have underflow check.
this patch have two change.
1. Change NR_CPU to nr_online_cpus()
vm_acct_memory() isn't fast-path. then cpumask_weight() calculation
isn't so expensive and the parameter for scalability issue should
consider number of _physical_ cpus. not theoretical maximum number.
2. Add under-flow check to meminfo_proc_show().
Almost field in /proc/meminfo have underflow check. but Committed_AS
is significant exeption.
it should do.
Reported-by: Dave Hansen <dave@...ux.vnet.ibm.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
---
fs/proc/meminfo.c | 2 ++
mm/swap.c | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
Index: b/fs/proc/meminfo.c
===================================================================
--- a/fs/proc/meminfo.c
+++ b/fs/proc/meminfo.c
@@ -36,6 +36,8 @@ static int meminfo_proc_show(struct seq_
si_meminfo(&i);
si_swapinfo(&i);
committed = atomic_long_read(&vm_committed_space);
+ if (committed < 0)
+ committed = 0;
allowed = ((totalram_pages - hugetlb_total_pages())
* sysctl_overcommit_ratio / 100) + total_swap_pages;
Index: b/mm/swap.c
===================================================================
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -519,7 +519,7 @@ EXPORT_SYMBOL(pagevec_lookup_tag);
* We tolerate a little inaccuracy to avoid ping-ponging the counter between
* CPUs
*/
-#define ACCT_THRESHOLD max(16, NR_CPUS * 2)
+#define ACCT_THRESHOLD max_t(long, 16, num_online_cpus() * 2)
static DEFINE_PER_CPU(long, committed_space);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists