linux-kernel - Re: [PATCH] mm: show stats for non-default hugepage sizes in /proc/meminfo

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 13 Nov 2017 16:33:05 +0000
From:   Roman Gushchin <guro@...com>
To:     Michal Hocko <mhocko@...nel.org>
CC:     <linux-mm@...ck.org>, Andrew Morton <akpm@...ux-foundation.org>,
        Johannes Weiner <hannes@...xchg.org>,
        Mike Kravetz <mike.kravetz@...cle.com>,
        "Aneesh Kumar K.V" <aneesh.kumar@...ux.vnet.ibm.com>,
        Andrea Arcangeli <aarcange@...hat.com>, <kernel-team@...com>,
        <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] mm: show stats for non-default hugepage sizes in
 /proc/meminfo

On Mon, Nov 13, 2017 at 05:11:02PM +0100, Michal Hocko wrote:
> On Mon 13-11-17 16:03:02, Roman Gushchin wrote:
> > Currently we display some hugepage statistics (total, free, etc)
> > in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> > 
> > If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> > /proc/meminfo output can be confusing, as non-default sized hugepages
> > are not reflected at all, and there are no signs that they are
> > existing and consuming system memory.
> 
> Yes this sucks but we do have per numa node per h-state stats in sysfs
> already /sys/devices/system/node/node*/hugepages
> 
> I know it is another source of the information but is there any reason
> you cannot use it?

Hi, Michal!

In my case, I didn't know in advance, that hugetlb pages are preallocated,
and spent some time trying to find "magically disappeared" several Gb of memory,
which are not reflected in any /proc/[meminfo,vmstat] counters.

IMO, /proc/meminfo should give a user a high-level overview of memory usage
in the system, without a need to look into other places. Of course, we always
have some amount of unaccounted memory, but it shouldn't be measured in Gb,
as in this case.

If you're worried about adding counters which will be 0 most of the time
for most users, we can print them conditionally, only if total number of
corresponding pages is not 0.

Thanks!