linux-kernel - Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20161021111732.GR14023@dastard>
Date:   Fri, 21 Oct 2016 22:17:32 +1100
From:   Dave Chinner <david@...morbit.com>
To:     Vlastimil Babka <vbabka@...e.cz>
Cc:     Michal Hocko <mhocko@...nel.org>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Johannes Weiner <hannes@...xchg.org>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Hugh Dickins <hughd@...gle.com>,
        David Rientjes <rientjes@...gle.com>
Subject: Re: [RFC] fs/proc/meminfo: introduce Unaccounted statistic

On Fri, Oct 21, 2016 at 09:25:10AM +0200, Vlastimil Babka wrote:
> On 10/21/2016 12:59 AM, Dave Chinner wrote:
> >On Thu, Oct 20, 2016 at 03:33:58PM +0200, Michal Hocko wrote:
> >>On Thu 20-10-16 14:11:49, Vlastimil Babka wrote:
> >>[...]
> >>> Hi, I'm wondering if people would find this useful. If you think it is, and
> >>> to not make performance worse, I could also make sure in proper submission
> >>> that values are not read via global_page_state() multiple times etc...
> >>
> >>I definitely find this information useful and hate to do the math all
> >>the time but on the other hand this is quite fragile and I can imagine
> >>we can easily forget to add something there and provide a misleading
> >>information to the userspace. So I would be worried with a long term
> >>maintainability of this.
> >
> >This will result in valid memory usage by subsystems like the XFS
> >buffer cache being reported as "unaccounted". Given this cache
> >(whose size is shrinker controlled) can grow to gigabytes in size
> >under various metadata intensive workloads, there's every chance
> >that such reporting will make users incorrectly think they have a
> >massive memory leak....
> 
> Is the XFS buffer cache accounted (and visible) somewhere then? I'd
> say getting such large consumers to become visible on the same level
> as others would be another advantage...

It's handles are visible via the xfs_buf slab cache. By the time
you've got enough memory in the buffer cache for it to be noticed,
the xfs_buf slab is near the top of the list in slabtop.

Of course, because of the crazy way slub names caches, this can be
impossible to find because there isn't a "xfs_buf" slab cache that
shows up in slabtop. It'll end being called something like
"mnt_cache"....

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com