linux-kernel - Re: [PATCH] proc/meminfo: add MemKernel counter

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190522170342.GA11077@tower.DHCP.thefacebook.com>
Date:   Wed, 22 May 2019 17:03:48 +0000
From:   Roman Gushchin <guro@...com>
To:     Konstantin Khlebnikov <khlebnikov@...dex-team.ru>
CC:     Michal Hocko <mhocko@...nel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Vlastimil Babka" <vbabka@...e.cz>,
        Johannes Weiner <hannes@...xchg.org>
Subject: Re: [PATCH] proc/meminfo: add MemKernel counter

On Wed, May 22, 2019 at 07:09:22PM +0300, Konstantin Khlebnikov wrote:
> On 22.05.2019 18:52, Michal Hocko wrote:
> > On Wed 22-05-19 17:40:09, Konstantin Khlebnikov wrote:
> > > Some kinds of kernel allocations are not accounted or not show in meminfo.
> > > For example vmalloc allocations are tracked but overall size is not shown
> > > for performance reasons. There is no information about network buffers.
> > > 
> > > In most cases detailed statistics is not required. At first place we need
> > > information about overall kernel memory usage regardless of its structure.
> > > 
> > > This patch estimates kernel memory usage by subtracting known sizes of
> > > free, anonymous, hugetlb and caches from total memory size: MemKernel =
> > > MemTotal - MemFree - Buffers - Cached - SwapCached - AnonPages - Hugetlb.
> > 
> > Why do we need to export something that can be calculated in the
> > userspace trivially? Also is this really something the number really
> > meaningful? Say you have a driver that exports memory to the userspace
> > via mmap but that memory is not accounted. Is this really a kernel
> > memory?
> > 
> 
> It may be trivial right now but not fixed.
> Adding new kinds of memory may change this definition.

Right, and it's what causes me to agree with Michal here, and leave it
to the userspace calculation.

The real meaning of the counter is the size of the "gray zone",
basically the memory which we have no clue about.

If we'll add accounting of some new type of memory, which now in this
gray zone (say, xfs buffers), we probably should exclude it too.
And this means that definition of this counter will change.

So IMO the definition is way too implementation-defined to be a part
of procfs API.