linux-kernel - Re: Process memory accounting (cgroups) accuracy

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YN7XgzB4bE2K9int@dhcp22.suse.cz>
Date:   Fri, 2 Jul 2021 11:08:19 +0200
From:   Michal Hocko <mhocko@...e.com>
To:     Krzysztof Kozlowski <krzysztof.kozlowski@...onical.com>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Johannes Weiner <hannes@...xchg.org>,
        Vladimir Davydov <vdavydov.dev@...il.com>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, cgroups@...r.kernel.org
Subject: Re: Process memory accounting (cgroups) accuracy

On Fri 02-07-21 09:50:11, Krzysztof Kozlowski wrote:
[...]
> The questions: How accurate are now the cgroup counters?

The precision depends on the number of CPUs the workload is running on
as we do a per-cpu charge caching to optimize the accounting. This is
MEMCG_CHARGE_BATCH (32) pages currently. You can learn more by checking
try_charge function (mm/memcontrol.c).

> I understood they should charge only pages allocated by the process, so
> why mmap(4 kB) causes max_usage_in_bytes=132 kB?

Please note that kernel allocations (marked by __GFP_ACCOUNT) are
accounted as well so this is not only about mmaped memory.

> Why mmap(4 MB) causes max_usage_in_bytes=4 MB + 34 pages?

The specific number will depend on the executing - e.g. use up all but 3
pages from CPU0 batch and have 31 pages on another cpu.

> What is being accounted there (stack guards?)?
> 
> Or maybe the entire LTP test checking so carefully memcg limits is useless?

Well, I haven't really checked details of those tests and their
objective but aiming for an absolute precision is not really something
that is very useful IMHO. We are very likely to do optimizations like
the one mentioned above as the runtime tends to be much more important
than to-the-page precision.

Hope this clarifies this a bit.
-- 
Michal Hocko
SUSE Labs